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Abstract 

The  three  mam  objectives  of  this  research  were  to  identify  the  actual  reasons  for 
schedule  problems  across  large  Air  Force  system  development  efforts,  to  quantify  the 
importance  of  each  category  of  reasons  in  terms  of  frequency  and  severity,  and  to 
demonstrate  that  the  reasons  for  schedule  problems  are  not  program  unique,  but  are 
common  across  system  development  efforts. 

To  this  end,  this  thesis  contains  a  categorization  and  analysis  of  549  reasons  for 
schedule  difficulties  on  22  large  Air  Force  Engineering  and  Manufacturing  Development 
(EMD)  programs  from  1981  to  1994.  These  aircraft,  missile,  aircraft  equipment,  aircraft 
upgrade,  and  simulator  programs  had  contract  values  ranging  from  $40M  to  over  $10B. 
All  reasons  were  extracted  from  narrative  explanations  of  negative  schedule  variances 
contained  in  contractor  generated  Cost  Performance  Reports  (CPRs). 

Reasons  for  schedule  problems  were  placed  into  categories,  and  categories  were 
ranked  by  frequency  of  problems,  total  schedule  variance  (in  dollars),  and  total  schedule 
variance  (in  work  days).  Seven  categories  (technical  problems,  late  subcontractors, 
manufacturing  problems,  design  changes,  late  data,  contracting,  and  staffing)  accounted 
for  49  percent  of  the  frequency,  57  percent  of  the  schedule  variance  (in  dollars),  and  49 
percent  of  the  schedule  variance  (in  work  days). 


viii 


WHY  SCHEDULES  SLIP: 


ACTUAL  REASONS  FOR  SCHEDULE  PROBLEMS 

ACROSS 

LARGE  AIR  FORCE  SYSTEM  DEVELOPMENT  EFFORTS 


I:  Introduction 


General  Issue 

Simply  put,  large  defense  systems  take  too  long  to  develop  (Drezner  and  Smith, 
1990:i,l).  Following  project  initiation,  a  typical  system  spends  between  one  and  six  years 
in  a  preliminary  design  and  prototype  hardware  phase.  If  approved  for  development,  an 
additional  three  to  ten  years  will  typically  pass  before  the  first  system  is  ready  for  delivery 
to  an  operational  unit  (Drezner  and  Smith,  1990:8-13).  Thus,  it  can  take  as  many  as 
sixteen  years  before  a  validated  system  concept  is  developed  into  an  operational  military 
capability.  In  today’s  environment  of  advancing  technology  and  changing  national  security 
concerns,  the  current  practice  of  fielding  1979  weapon  system  concepts  in  1995  systems 
simply  does  not  make  sense. 
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The  idea  that  large  defense  system  development  takes  too  long  is  further  supported 
by  the  fact  that  system  development  projects  are  typically  late  even  by  their  own 
schedules.  Specifically,  the  average  defense  system  development  effort  requires  one-third 
more  time  to  complete  than  was  originally  scheduled  (Augustine,  1983:115;  Drezner  and 
Smith,  1990:44). 

There  are  four  consequences  of  overly  lengthy  system  development  efforts.  First, 
since  a  system’s  design  tends  to  be  ‘locked  in”  early  in  development,  the  longer  it  takes  to 
field  the  system,  the  more  likely  the  system  will  be  based  on  old  —  even  obsolete  — 
technology.  Such  systems  reduce  the  effectiveness  of  operational  units,  which  rely  on 
superior  weapon  systems  to  provide  them  with  a  combat  advantage  over  enemy  forces 
(Drezner  and  Smith,  1990:1,1).  Second,  a  longer  development  effort  increases  the  cost  of 
a  system  in  terms  of  inflation  and  overhead  (Drezner  and  Smith,  1990: 1).  Third,  a  longer 
development  effort  provides  more  opportunities  for  introducing  technical  changes  into, 
and  new  requirements  for,  the  system  (Drezner  and  Smith,  1990: 1).  These  changes 
further  lengthen  the  project  and  often  lead  to  technical  problems.  Fourth,  the  longer  a 
development  effort,  the  more  likely  it  is  to  be  canceled  prior  to  first  system  delivery 
(Augustine,  1983:203-204).  One  reason  this  last  phenomenon  may  occur  is  because 
managers  tend  to  associate  schedule  problems  with  project  failure,  and  projects  perceived 
to  be  failing  are  prime  candidates  for  cancellation  (Pinto  and  Mantel,  1990:273).  In  any 
case,  a  canceled  development  effort  often  represents  a  “lose-lose”  proposition  in  which 
money  is  wasted  and  operational  needs  remain  unfulfilled. 
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Thus,  as  was  stated  in  the  Packard  Commission  report  on  defense  management, 
the  length  of  the  defense  system  development  process  is  “a  central  problem  from  which 
most  other  acquisition  problems  stem”  (The  President’s  Blue  Ribbon  Commission  on 
Defense  Management,  1986:47). 

Specific  Problem 

Clearly,  there  is  a  need  for  improving  schedule  performance  in  the  development  of 
large  defense  systems.  Logically,  such  improvement  could  be  realized  by  first  identifying 
actual  reasons  for  schedule  problems  on  these  efforts,  then  taking  appropriate  corrective 
action  such  as  providing  supplemental  training,  changing  administrative  procedures,  or 
improving  techniques.  Unfortunately,  very  little  research  addresses  the  actual  reasons  for 
schedule  problems  on  defense  systems  development  efforts  (i.e.,  late  vendor  selection  or 
excessive  engineering  changes),  and  the  research  that  does  exist  categorizes  reasons  at  too 
high  a  degree  of  abstraction  to  enable  the  development  of  appropriate  corrective  actions. 
Without  knowledge  of  these  reasons,  any  attempt  to  improve  schedule  performance  is 
little  more  than  a  “shot  in  the  dark.” 

Thesis  Objectives 

The  objectives  of  this  thesis  are  to: 

(a)  identify  the  actual  reasons  for  schedule  problems  across  large  Air  Force  system 
development  efforts,  describing  those  reasons  at  a  level  of  detail  that  will 
allow  the  development  of  appropriate  corrective  actions; 
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(b)  quantify  the  importance  of  each  category  of  reasons,  in  terms  of  frequency  and 

severity,  in  order  to  determine  the  categories  of  reasons  most  and  least 
deserving  of  management  attention;  and, 

(c)  demonstrate  that  the  reasons  for  schedule  problems  are  not  program  unique, 

but  are  common  across  system  development  efforts,  therefore  schedule- 
related  lessons  learned  from  past  and  present  efforts  are  likely  to  be 
relevant  to  future  efforts. 

Scope 

This  thesis  examines  the  reasons  for  schedule  difficulties  on  22  Air  Force 
Engineering  and  Manufacturing  Development  (EMD)  programs  from  1981  to  1994. 

These  aircraft,  missile,  aircraft  equipment,  aircraft  upgrade,  and  simulator  programs  had 
contract  values  ranging  from  S40M  to  over  $10B.  All  reasons  were  extracted  from 
narrative  explanations  of  negative  schedule  variances  contained  in  contractor  generated 
Cost  Performance  Reports  (CPRs),  a  standard  deliverable  data  item  on  large  defense 
system  development  contracts.  Frequency  and  severity  data  for  each  category  of  reasons 
were  obtained  from  the  same  CPRs.  The  severity  data  was  based  on  schedule  variances 
associated  with  each  observed  reason  for  schedule  problems.  In  order  to  ensure  reasons 
were  obtained  across  programs,  data  for  each  program  were  taken  from  the  same  number 
of  CPRs  as  for  all  other  programs. 
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II:  Literature  Review 


Introduction 

The  literature  contains  very  little  research  dealing  directly  with  the  actual  reasons 
for  schedule  problems  across  large  defense  system  development  efforts.  The  literature 
does,  however,  contain  two  related  areas:  (1)  theorized  reasons  for  schedule  problems, 
and  (2)  reasons  for  schedule  problems  observed  on  actual  projects.  These  areas  will  be 
discussed  in  order,  followed  by  an  explanation  of  limitations  in  the  current  body  of 
knowledge. 

Potential  Reasons  for  Schedule  Problems 

Table  2-1  summarizes  some  potential  reasons  for  schedule  problems.  They  have 
been  synthesized  from  a  variety  of  sources  including:  contract  administration  and  materials 
management  books  (Contract  Administration,  1975;Riemer,  1968;Ammer,  1974); 
surveys  of  project  management  instructors,  practitioners,  and  headquarters  personnel 
(Ketchum  and  McKenzie,  1976;  Dunbar,  1980);  a  study  of  govemment/industry 
difficulties  (Montgomery,  1978);  a  literature  review  and  field  interviews  on  reasons  for 
construction  project  delays  (Dawkins,  1987),  and  an  examination  of  potential  trouble  areas 
on  ten  specific  weapon  system  projects  (Drezner  and  Smith,  1990).  Note  that  these 
reasons  are  based  on  expert  opinion,  rather  than  on  formal  studies  of  schedule  problems 
occurring  on  specific  programs. 
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TABLE  2-1:  Potential  Reasons  for  Schedule  Problems 


Planning 

Procedures 

Poor  Support  Planning 

Source  Selection 

Poor  Resource  Planning 

Poor  Quality  Control  Causing  Rework 

Poor  Financial  Planning 

Failure  to  Inspect  in  a  Timely  Manner 

Backlog  of  Orders 

Failure  to  Make  Timely  Progress  Payments 

Unrealistic  Overall  Schedule 

Joint  Service  Project  Management 

Late  Government  Furnished  Equipment 

Paperwork  Processing  Delays  (Reviews/ Approvals) 

Late  Receipt  of  Materials 

Poor  Inspection/ Acceptance  Criteria 

Poor  Management  Practices/Lack  of  Follow  Up 

Changes 

Failure  to  Coordinate  Multiple  Contractors 

Failure  to  Obtain  Proper  Approvals 

Poor  Management  of  Changes 

Slow  “Ramp  Up”  of  Resources 

Engineering  Change  Proposals 

Failure  to  Understand  the  Contract 

Design  Changes 

Failure  to  Communicate 

Inadvertent  Constructive  Changes 

Inadequate  Supervision 

Government  Directed  Changes  to  Work  Tempo 

Micromanaging  the  Contractor 

Execution 

External 

System  Integration 

Political  Influences 

Contractor  Motivation 

External  Guidance 

Technical  Difficulties 

Concept  Stability 

Labor  Problems 

External  Event 

Defective  Government  Furnished  Equipment 

Funding  Stability 

Default  of  Subcontractors 

Major  Requirements  Stability 

Program  Complexity  (Coordination  Burden) 
Contractor  Performance 

Program  Manager  Turnover 

Economic  Factors 

(Adapted  from  Ammer,  1974:437;  Contract  Administration,  1975:161-171;  Dawkins,  1987:7-8;  Drezner 
and  Smith,  1990:21-22;  Dunbar,  1980:40;  Ketchum  and  McKenzie,  1976:23;  Montgomery,  1978:30;  and 
Riemer,  1968:553) 


There  are  three  main  points  to  note  about  the  data  in  Table  2-1.  First,  in  spite  of 
the  variety  of  sources,  the  reasons  for  schedule  problems  are  fairly  consistent.  For 
example,  “major  requirements  stability”  is  mentioned  in  four  of  the  eight  sources  (Contract 
Administration,  1975;  Dawkins,  1987;  Drezner  and  Smith,  1990;  Ketchum  and  McKenzie, 
1976),  ‘late/defective  government  furnished  equipment”  is  mentioned  three  times 
(Dawkins,  1987;  Dunbar,  1980;  Montgomery,  1978),  and  ‘'poor  quality  control”  is 


6 


mentioned  twice  (Dawkins,  1987;  Montgomery,  1978).  Even  though  many  reasons  are 
only  mentioned  once,  their  similarity  suggests  that  the  actual  reasons  for  poor  schedule 
performance  on  large  system  development  projects  will  resemble  those  listed  in  Table  2-1. 
Second,  there  are  many  reasons  that  could  potentially  impede  schedule  performance. 

Table  2-1  contains  forty-three  reasons,  many  of  which  need  to  be  subdivided  into  still 
more  reasons  before  they  provide  useful  information  on  which  to  base  corrective  actions. 
Third,  the  literature  that  deals  with  theoretical  reasons  for  schedule  problems  fails  to 
provide  any  information  on  the  frequency  of  these  reasons  or  the  severity  of  their 
consequences.  This  is  significant,  because  it  is  impractical  to  implement  corrective  actions 
for  all  the  reasons  identified  in  Table  2-1.  Thus,  the  theoretical  literature  fails  to  provide 
guidance  as  to  which  reasons  most  deserve  attention. 

Reasons  for  Schedule  Problems  Observed  on  Actual  Projects 

Guidance  as  to  which  reasons  deserve  the  most  attention  improves  somewhat  in 
the  literature  dealing  with  observed  reasons  for  schedule  problems.  Specifically,  two 
studies  provide  reasons  for  poor  schedule  performance,  the  amount  of  delay  associated 
with  each  reason,  and  roughly  how  often  each  reason  is  encountered.  The  first  study, 
summarized  in  Table  2-2,  investigates  delays  in  the  development  often  major  weapon 
systems  and  therefore  applies  well  to  this  thesis.  Unlike  the  theorized  reasons  for  poor 
schedule  performance  presented  earlier,  the  significance  of  the  reasons  in  Table  2-2  has 
been  identified.  For  example,  “technical  difficulties”  account  for  more  poor  schedule 
performance  than  “external  events.” 
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TABLE  2-2:  Significance  of  Reasons  for  Schedule  Problems 
on  Ten  Major  Weapon  System  Development  Projects 


Reason 

Percent  of  Total  Delav* 

Technical  Difficulty 

29.8% 

5 

External  Guidance 

22.5% 

5 

Unknown  (could  not  identify) 

18.5% 

1 

Funding  Stability 

16.7% 

5 

External  Event 

7.6% 

4 

Contractor  Performance 

2.5% 

2 

Program  Complexity 

1.8% 

1 

Concept  Stability 

1.5% 

1 

Major  Requirements  Stability 

-1% 

2 

a  Total  delay  attributed  to  a  reason  (Adapted  from  Drezner  and  Smith,  1990:32-35) 


divided  by  total  delay  across  programs 
b  Reason  observed  on  this  number  of  projects 

There  are  two  difficulties  associated  with  applying  the  results  of  the  study  to  this 
thesis.  First,  the  reasons  listed  in  Table  2-2  are  too  abstract  to  serve  as  a  basis  for 
choosing  corrective  actions  designed  to  improve  schedule  performance.  For  example, 
how  does  one  actually  go  about  improving  “technical  difficulty”  or  “contractor 
performance”?  Unless  these  categories  can  be  subdivided  into  more  precise  reasons,  they 
are  of  limited  usefulness.  Second,  there  is  very  little  information  regarding  how  often  each 
reason  occurs.  Although  the  study  indicates  whether  or  not  a  reason  occurs  on  a 
development  project,  and  provides  the  amount  of  delay  associated  with  the  reason,  it  does 
not  identify  how  often  that  reason  is  observed  in  the  development  project  (Drezner  and 
Smith,  1990:32-34).  Thus,  important  information  for  determining  appropriate  corrective 
actions  to  remedy  poor  schedule  performance  is  missing  from  the  study. 

The  second  study,  summarized  in  Table  2-3,  examines  the  reasons  for  delay  on 
forty-eight  general  building  (aircraft  hangars,  military  personnel  housing,  instructional 
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facilities,  laboratories,  mo dification/ conversion/building  addition  projects,  office  buildings, 
and  warehouse  facilities)  construction  contracts  (Dawkins,  1987:44-45). 


TABLE  2-3:  Significance  of  Reasons  for  Schedule  Problems 
on  Forty-Eight  General  Building  Construction  Contracts 


Reason 

Percent  of  Total  Delav 

Number  of  Chances3 

Design  Error 

33.3% 

157 

Site  not  as  Expected/ 

27.2% 

130 

Unforeseen  Work 

Discretionary  Owner  Change 

18.7% 

50 

Time  Extension 

14.3% 

27 

Mandatory  Owner  Change 

5.4% 

21 

Claims  Settlement 

1.1% 

1 

a  Total  number  of  contract  changes  associated  with  each  reason  (Adapted  from  Dawkins,  1987:66,69) 


Although  there  are  many  differences  between  building  construction  and  major 
defense  system  development,  there  are  also  similarities.  For  example,  “design  error”  in 
Table  2-3  (construction)  could  easily  refer  to  “technical  difficulty”  in  Table  2-2  (weapon 
systems).  Likewise,  “mandatory  owner  change”  in  Table  2-3  (construction)  could  easily 
refer  to  “major  requirements  stability”  in  Table  2-2  (weapon  systems).  Thus  both  studies 
are  useful  in  identifying  the  general  categories  of  reasons  for  poor  schedule  performance 
and  in  estimating  the  delays  resulting  from  them. 

The  construction  study,  however,  has  a  problem.  As  in  Table  2-2  (weapon 
systems),  the  reasons  for  poor  schedule  performance  in  Table  2-3  (construction)  are  still 
too  abstract  to  serve  as  a  basis  for  choosing  corrective  actions  designed  to  improve 
schedule  performance.  The  author  solved  this  problem  by  dividing  the  reasons  listed  in 
Table  2-3  into  more  specific  sub-categories.  For  example,  a  “design  error”  may  fall  into  a 
subcategory  called  “electrical”  or  “internal  architecture”  (Dawkins,  1987:66).  If  a  large 
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percentage  of  reasons  for  poor  schedule  performance  fall  into  the  “design  error  - 
electrical”  subcategory,  electrical  designers  may  need  to  receive  supplemental  training  in 
this  area.  While  the  subcategories  are  construction-specific  and  would  not  provide 
appropriate  guidance  for  large  defense  system  development  projects,  this  method  seems 
very  useful  for  providing  detailed  knowledge  about  reasons  for  poor  schedule  performance 
in  general. 

Reasons  Why  Previous  Research  on  Schedule  Problems  is  Inadequate 

As  presented  earlier,  the  literature  suggests  forty-three  potential  reasons  for  poor 
schedule  performance  (See  Table  2-1).  It  also  attempts  to  identify  the  consequences 
associated  with  fourteen  reasons  observed  on  actual  projects  (See  Tables  2-2,2-3). 
Unfortunately,  it  neither  describes  the  reasons  for  schedule  problems  at  a  level  of  detail 
appropriate  for  determining  corrective  actions,  nor  does  it  quantify  categories  of  reasons 
in  a  manner  that  allows  management  to  determine  which  reasons  are  the  most  deserving  of 
immediate  attention.  There  are  three  probable  reasons  why  previous  research  has  not 
provided  this  information. 

First,  there  is  a  prevailing  attitude  that  poor  schedule  performance  only  occurs 
when  an  effort  is  completed  later  than  its  scheduled  date  (for  example,  see  Drezner, 

1990: 17).  If  this  were  truly  the  case,  the  easiest  way  to  improve  schedule  performance 
would  be  to  lengthen  the  schedule.  This  is,  in  fact,  what  various  schedule  estimating 
methodologies  accomplish  (for  example,  Boyd  and  Mundt,  1993;  Harmon  and  Ward, 

1990;  Harmon,  Ward,  and  Palmer,  1989).  By  basing  schedule  predictions  on  historical 
data,  these  methodologies  assume  the  mistakes  and  inefficiencies  of  the  past  will  continue 
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into  the  future.  When  researchers  take  this  stance,  they  fail  to  investigate  ways  to  improve 
project  execution.  They  also  fail  to  look  for  signposts,  such  as  the  reasons  for  poor 
schedule  performance,  that  would  identify  potential  areas  to  improve. 

Second,  there  is  a  belief  that  schedule  performance  on  large  defense  system 
development  projects  is  controlled  primarily  by  factors  external  to  the  contractor  and 
government  project  office.  In  fact,  the  previously  mentioned  study  often  major  weapon 
system  development  projects  found  that  five  of  the  projects  had  between  seventy  and  one- 
hundred  percent  of  their  delays  caused  by  external  factors.  This  result,  however,  must  be 
viewed  with  some  degree  of  skepticism,  since  three  of  the  ten  projects  had  over  seventy 
percent  of  their  delay  caused  by  internal  factors  (Drezner  and  Smith,  1990:34-37). 

Further,  studies  in  both  the  construction  industry  and  in  weapon  system  production  found 
that  a  majority  of  poor  schedule  performance  is  due  to  the  internal  factor  of  poor 
management  practice  (Newmann,  1983:32;  Dunbar,  1980: 104).  Finally,  a  recent  survey  of 
small  architecture  and  engineering  firms  found  that  as  the  use  of  project  management  “best 
practices”  increases,  projects  are  more  likely  to  meet  or  exceed  their  schedule  objectives 
(Anderson  and  Tucker,  1994:40).  Thus,  there  is  a  fair  amount  of  evidence  that  the 
government  project  office  and  the  contractor  for  a  large  defense  system  development 
project  can  improve  their  schedule  performance  through  improving  the  effectiveness  of 
their  operations.  Research  into  the  reasons  for  schedule  problems  is  necessary  in  order  to 
pinpoint  where  improvements  need  to  be  made. 

Third,  there  is  a  widespread  belief  that  the  reasons  for  schedule  problems  on  one 
project  will  not  apply  to  the  next  project.  Therefore,  research  into  reasons  for  schedule 
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problems  is  of  little  use.  This  view  is  understandable  given  the  unique  nature  of  projects. 

In  his  project  management  text,  Nicholas  states  that  “a  project  is  a  one  time  activity,  never 
to  be  exactly  repeated  again”  (Nicholas,  1990:4).  Thus,  it  is  not  surprising  that  project 
managers  tend  to  see  the  reasons  for  various  project  failures,  including  poor  schedule 
performance,  as  isolated  incidents  that  are  not  generalizable  across  projects  (Pinto  and 
Mantel,  1990:269).  This  perception  is  perpetuated  through  a  lack  of  carry-over 
experience  from  one  project  to  the  next  due  to  factors  ranging  from  the  breakup  of  project 
teams  at  the  conclusion  of  a  project,  to  the  lack  of  training  designed  to  preserve  lessons 
previously  learned  for  future  projects  (Bitner,  1985:73).  Only  through  further  research 
can  this  perception  be  altered.  Just  as  Ketchum  and  McKenzie,  in  their  formulation  of 
fifteen  weapon  system  acquisition  case  studies,  found  that  “the  method  of  dealing  with 
acquisition  management  problems  may  differ,  but  the  problems  themselves  seem  to  remain 
essentially  the  same,”  (Ketchum  and  McKenzie,  1976:3 1)  the  study  of  reasons  for 
schedule  problems  will  most  likely  reveal  similarities  in  these  reasons  across  weapon 
systems.  These  reasons  can  then  be  used  to  determine  generally  applicable  solutions  to  the 
problem  of  poor  schedule  performance. 

Conclusion 

The  literature  does  not  contain  a  single  study,  or  collection  of  studies,  that:  (a) 
identifies  actual  reasons  for  schedule  problems  across  large  Air  Force  system  development 
efforts,  describing  those  reasons  at  a  level  of  detail  that  allows  the  development  of 
appropriate  corrective  actions;  (b)  quantifies  the  importance  of  each  reason  in  terms  of  its 
frequency,  and  severity  in  a  manner  that  allows  the  determination  of  the  categories  of 
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reasons  most  and  least  deserving  of  management  attention;  and,  (c)  demonstrates  that  the 
actual  reasons  for  schedule  problems  are  not  program  unique,  but  are  common  across 
system  development  efforts.  This  thesis  will  build  upon  the  related  research  summarized 
in  this  literature  review,  and  expand  the  existing  body  of  knowledge  dealing  with  the 
actual  reasons  for  schedule  problems  in  order  to  satisfy  (a),  (b),  and  (c)  above. 
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Ill:  Methodology 


Introduction 

This  research  had  three  main  objectives.  The  first  was  to  identify  the  actual 
reasons  for  schedule  problems  across  large  Air  Force  system  development  efforts, 
describing  those  reasons  at  a  level  of  detail  that  will  allow  the  development  of  appropriate 
corrective  actions.  The  second  was  to  quantify  the  importance  of  each  category  of 
reasons,  in  terms  of  frequency  and  severity,  in  order  to  determine  the  categories  of  reasons 
most  and  least  deserving  of  management  attention.  The  third  was  to  demonstrate  that  the 
reasons  for  schedule  problems  are  not  program  unique,  but  are  common  across  system 
development  efforts,  therefore  schedule-related  lessons  learned  from  past  and  present 
efforts  are  likely  to  be  relevant  to  future  efforts.  This  chapter  provides  a  detailed 
description  of  the  manner  in  which  the  research  was  conducted  in  order  to  satisfy  the 
above  objectives.  It  discusses  the  overall  research  approach,  the  appropriateness  of  the 
data  source,  the  pilot  study  conducted  to  determine  the  data  collection  methodology,  the 
specific  data  to  be  collected  along  with  the  rationale  for  choosing  those  particular  data,  the 
use  of  schedule  variance  in  this  study,  the  sampling  frame,  the  sampling  process,  the  data 
collection  process,  and  the  method  of  data  analysis. 

Overall  Research  Approach 

Because  studies  identifying  and  quantifying  the  reasons  for  schedule  problems  on 
large  defense  system  development  efforts  do  not  currently  exist,  this  research  is  intended 
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to  provide  the  missing  foundation  upon  which  project  managers  can  build  schedule 
problem  insight  and  researchers  can  base  investigations  into  more  specific  schedule-related 
management  questions.  This  research,  then,  is  a  descriptive  study  (Emory  and  Cooper, 
1991: 148)  in  which  the  goal  is  to  provide  information  on  observed  reasons  for  schedule 
problems  on  defense  system  development  efforts  that  will  be  useful  in  preventing  those 
problems  in  the  future. 

Although  this  research  is  neither  focused  on  determining  causal  relationships  nor 
on  answering  specific  management  questions,  it  is  still  a  formal  study  (Emory  and  Cooper, 
1991: 140)  in  that  it  has  well-defined  objectives  related  to  identifying  and  quantifying  the 
reasons  for  schedule  problems. 

In  order  to  identify  and  quantify  the  reasons  for  schedule  problems  in  an  unbiased 
manner  this  research  uses  an  observational,  ex  post  facto  approach  (Emory  and  Cooper, 
1991: 140- 141)  in  which  all  data  are  taken  from  official  reports.  Because  these  reports  are 
prepared  by  people  familiar  with  the  schedule  problems  and  are  recorded  soon  after  the 
problems  occur,  their  use  helps  to  ensure  that  reasons  for  the  problems  are  neither 
distorted  nor  omitted.  If  this  approach  were  not  taken,  it  would  be  likely  that  only 
reasons  associated  with  the  more  memorable  or  more  recent  schedule  problems  would  be 
captured  (Emory  and  Cooper,  1991:402). 

Another  fundamental  aspect  of  this  research  is  that  it  examines  many  system 
development  efforts,  rather  than  one  or  two  in  depth.  This  approach  helps  to  ensure  that 
the  results  of  this  research  reflect  the  variety  of  reasons  for  schedule  problems 
encountered  across  system  development  efforts,  and  that  these  results  will  most  likely 
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apply  to  similar  development  efforts  not  specifically  studied.  This  approach  also  satisfies 
the  research  objective  of  demonstrating  that  the  reasons  for  schedule  problems  are  not 
program  unique,  but  are  common  across  programs. 

Finally,  this  research  is  conducted  using  a  longitudinal  approach  (Emory  and 
Cooper,  1991: 141)  in  that  it  examines  the  reasons  for  schedule  problems  on  defense 
system  development  efforts  over  time.  Compared  to  a  cross-sectional  study,  which 
considers  observations  within  a  “‘snapshot’  of  one  point  in  time,”  (Emory  and  Cooper, 
1991: 141)  this  approach  provides  far  more  potential  data.  Given  that  system  development 
efforts  may  experience  different  types  of  problems  during  various  stages  of  development, 
this  approach  most  likely  also  provides  a  better  representation  of  the  reasons  for  problems 
experienced  within  the  development  phase  as  a  whole. 

Appropriateness  of  the  Data  Source 

The  data  source  for  this  research  is  the  Cost  Performance  Report  (CPR),  a 
standard  deliverable  data  item  typically  required  on  large  defense  system  development 
contracts.  The  CPR  is  an  appropriate  data  source  for  the  following  reasons.  First,  CPRs 
contain  information  regarding  both  the  reasons  for  schedule  problems  and  the  severity  of 
those  problems.  Thus,  they  contain  the  type  of  information  needed  to  satisfy  the  research 
objectives.  Second,  CPRs  have  been  used  on  large  defense  system  development  contracts 
for  many  years.  Thus,  they  provide  a  reasonably  consistent  source  of  data  on  a  wide 
variety  of  development  efforts  for  the  duration  of  those  efforts.  Third,  CPRs  are  prepared 
by  the  developing  contractor  at  roughly  the  same  time  as  the  schedule  problems  being 
reported.  This  improves  the  credibility  of  the  data  because  it  is  recorded  by  knowledgable 
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individuals  who  are  not  relying  heavily  on  memory  to  produce  the  reports.  Fourth,  CPRs 
are  generated  monthly.  This  helps  to  ensure  the  data  are  at  the  right  resolution  to  help 
identify  the  reasons  for  schedule  problems  at  a  level  of  detail  that  allows  the  formulation  of 
appropriate  corrective  actions.  If  the  CPR  were  less  frequent,  the  data  may  be  too 
aggregated  to  suggest  corrective  actions.  Regular  monthly  reports  also  facilitate  the 
sampling  of  data  across  development  efforts.  Fifth,  CPRs  for  large  Air  Force  system 
development  efforts  managed  by  the  Aeronautical  Systems  Center  (ASC)  are  readily 
available  in  the  ASC  Cost  Library.  This  is  significant,  because  consistently  recorded  and 
archived  schedule  information  on  defense  system  development  efforts  is  difficult,  if  not 
impossible  to  find  otherwise. 

Pilot  Study 

In  order  to  determine  data  availability,  the  most  appropriate  data  to  collect,  and  the 
most  effective  method  for  data  collection,  a  pilot  study  involving  77  CPRs  from  the  C-17 
system  development  effort  was  conducted.  This  study  involved  obtaining  CPRs  from  the 
ASC  Cost  Library,  recording  narrative  and  numerical  data  associated  with  negative 
schedule  variances  (schedule  variances  will  be  explained  in  the  next  section),  and 
evaluating  the  data  and  the  data  collection  process  in  terms  of  the  above  pilot  study 
objectives.  The  pilot  study  supported  three  conclusions.  First,  it  was  determined  that 
sufficient  data  existed  in  CPRs  to  support  this  thesis  effort.  Second,  although  CPRs 
contain  a  wide  range  of  cost  and  schedule  data  presented  in  a  variety  of  ways,  only  a  small 
subset  of  these  data  were  relevant  to  this  research.  The  specific  data  subset,  described  in 
the  next  section,  was  chosen  based  on  insights  gained  during  the  pilot  study.  Third,  data 
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collection  for  the  thesis  effort  had  the  potential  to  be  extremely  time  consuming.  For  this 
reason,  a  sampling  methodology  (described  in  a  later  section)  would  have  to  be  adopted. 
Data  Selection 

Based  on  the  pilot  study  described  in  the  previous  section,  specific  types  of  data 
were  selected  from  the  CPRs  for  use  in  this  thesis.  This  section  both  identifies  these  data, 
and  provides  justifications  for  their  use. 

Data  Identifying  Schedule  Problems.  In  a  CPR,  each  time  the  schedule  variance 
(a  concept  that  will  be  explained  shortly)  exceeds  a  predetermined  threshold,  the 
contractor  is  required  to  explain  that  variance  in  narrative  form  Although  thresholds  vary 
from  contract  to  contract,  and  can  even  vary  during  the  length  of  a  single  contract,  the 
threshold  used  on  any  given  CPR  has  been  chosen  to  ensure  that  only  significant  variances 
are  reported.  The  fact  that  the  CPR  only  explains  variances  significant  to  the 
corresponding  system  development  effort  increases  the  relevance  of  studies  using  CPRs  as 
a  data  source.  The  narratives  that  accompany  negative  schedule  variances  (which 
correspond  to  behind  schedule  conditions)  are  the  sole  source  of  reasons  for  schedule 
problems  on  large  defense  system  development  efforts  contained  in  this  research. 

Measures  of  Schedule  Problems  Severity.  In  addition  to  the  narrative  data, 
which  identifies  the  reason  for  a  given  schedule  problem,  numerical  data  must  also  be 
collected  to  quantify  the  severity  of  that  problem  While  frequency  data  is  obtained  merely 
by  counting  the  number  of  observed  reasons  for  schedule  problems,  the  magnitude  of  the 
problems  associated  with  the  reasons  is  obtained  by  collecting  the  schedule  variance  and 
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the  Budgeted  Cost  of  Work  Scheduled  (BCWS)  for  each  collected  reason  (these  terms 
will  be  explained  shortly). 

Before  explaining  what  is  meant  by  schedule  variance  and  BCWS,  and  the 
appropriateness  of  these  measures  to  this  research,  there  are  two  important  elements  of 
data  selection  to  be  discussed:  the  use  of  current  month  rather  than  cumulative  data,  and 
the  use  of  data  associated  with  Work  Breakdown  Structure  (WBS)  elements  rather  than 
data  associated  with  functional  areas. 

Use  of  Current  Month  Data.  Although  CPRs  contain  both  cumulative  and 
current  month  data,  only  current  month  data  was  collected.  Part  of  the  reason  for  doing 
this  is  to  avoid  double- counting  schedule  problems  when  sampling  multiple  CPRs  on  a 
given  development  effort.  Another  reason  for  collecting  only  current  month  data  is  that 
narratives  associated  with  cumulative  schedule  variances  generally  contain  many  more 
reasons  for  the  variance  than  do  current  month  narratives.  The  more  reasons  per  schedule 
variance  the  more  difficult  it  is  to  accurately  separate  and  quantify  the  magnitude  of  the 
schedule  problems  associated  with  each  reason.  Also,  cumulative  data  tends  to  be  more 
sensitive  to  schedule  variances  early  in  an  effort,  and  less  sensitive  towards  the  end  of  an 
effort.  This  is  because  explanations  are  only  generated  when  schedule  variance  exceeds  a 
predetermined  threshold.  This  threshold  may  be  in  absolute  terms  (dollars),  relative  terms 
(percent),  or  a  combination  of  both  (AFSCP  173-4,  1989:  Para.  4-3(c)(3)(a)).  Ifthe 
threshold  is  defined  in  relative  terms,  a  given  variance  early  in  an  effort  may  be  of  the  same 
magnitude  as  a  variance  late  in  the  effort,  but  only  the  early  variance  will  exceed  the 
threshold  and  be  reported.  This  is  because  at  the  time  of  the  earlier  variance,  less  work 
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was  scheduled  to  be  completed  than  at  the  time  of  the  later  variance,  and  as  a  proportion, 
causes  the  same  amount  of  variance  to  appear  larger. 

Use  of  Data  Reported  Against  WBS  Elements.  Although  CPRs  identify 
schedule  problems  both  by  Work  Breakdown  Structure  (WBS)  element  and  by  functional 
area,  only  data  corresponding  to  WBS  elements  were  collected.  This  approach  was  taken 
because  in  the  C-17  pilot  study,  the  reasons  for  schedule  problems  were  better  separated 
when  reported  against  WBS  elements.  Functional  area  explanations  tended  to  contain 
many  reasons  per  reported  variance,  which,  as  was  explained  in  the  discussion  of  current 
month  versus  cumulative  data,  is  an  undesirable  attribute.  Also,  as  will  be  explained 
shortly,  schedule  variance  is  measured  based  on  progress  towards  completing  tasks 
defined  by  WBS  elements.  Because  the  determination  of  schedule  variance  is  more  closely 
tied  to  WBS  elements  than  to  functional  elements,  the  collection  of  data  organized  by 
WBS  element  rather  than  by  functional  area  is  appropriate. 

The  Use  of  Schedule  Variance  in  this  Study 

This  study  relies  on  the  use  of  schedule  variance  to  quantify  the  severity  of 
schedule  problems  on  large  defense  system  development  efforts.  In  order  to  appreciate 
the  usefulness  and  limitations  of  schedule  variance  as  a  measure,  it  is  important  to 
understand  what  schedule  variance  is,  and  how  it  is  calculated  on  a  CPR. 

Schedule  variance  measures  the  difference  between  progress  made  over  a  given 
period  of  time  (in  the  case  of  this  research,  a  month)  and  progress  scheduled  to  occur 
during  that  same  period  of  time  (The  AFSC  Cost  Estimating  Handbook:  Para.  5.4.2).  If 
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an  effort  is  exactly  on  schedule,  the  schedule  variance  is  zero.  A  negative  schedule 
variance,  however,  indicates  that  the  effort  is  behind  schedule. 

In  order  to  calculate  schedule  variance,  both  the  amount  of  progress  scheduled  to 
occur  and  the  amount  of  progress  that  actually  did  occur  must  be  defined.  The  amount  of 
progress  scheduled  to  occur  during  a  given  period  is  known  as  the  Budgeted  Cost  of 
Work  Scheduled  (BCWS)  (Nicholas,  1990:385).  The  BCWS  is  typically  calculated  at  the 
start  of  an  effort  by  defining  the  tasks  required  to  complete  the  effort  and  estimating  the 
amount  of  work  (in  dollars)  involved  in  each  task.  The  amount  of  work  estimated  for 
each  task  is  then  distributed  over  time,  based  on  the  schedule  for  the  effort,  to  form  a 
time-phased  budget  (Nicholas,  1990:355-356).  The  amount  of  work  in  this  time-phased 
budget  that  is  planned  to  occur  during  a  given  period  is  that  period’s  BCWS.  The  current 
month  BCWS,  then,  represents  the  amount  of  work  that  would  occur  during  that  month  if 
the  effort  were  on  schedule. 

Because  large  defense  system  development  efforts  are  rarely,  if  ever,  precisely  on 
schedule,  another  measure,  capturing  the  actual  progress  made  during  the  current  month, 
is  necessary.  This  measure,  known  as  the  Budgeted  Cost  of  Work  Performed  (BCWP)  or 
“earned  value,”  only  takes  credit  for  actual  progress  towards  completing  the  effort  (The 
AFSC  Cost  Estimating  Handbook:  Para.  5.4.2).  The  difference  between  the  BCWP  and 
the  BCWS  in  a  given  month  represents  the  difference  between  the  amount  of  progress 
actually  made  during  that  month  and  the  amount  of  progress  scheduled  to  be  made  during 
that  month.  This  difference,  known  as  the  monthly  schedule  variance,  is  negative  if  less 
progress  occurred  than  was  scheduled  (Nicholas,  1990:387-388).  By  collecting  the 
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negative  schedule  variance  associated  with  a  schedule  problem,  the  magnitude  of  that 
problem  in  terms  of  progress  not  made  can  be  quantified. 

Note,  however,  that  schedule  variance  is  measured  in  dollars  rather  than  in  time. 
This  is  because  it  is  a  measure  of  deviation  from  scheduled  progress,  and  progress  is 
measured  against  the  time-phased  estimated  cost  of  the  tasks  comprising  the  development 
effort.  Although  negative  schedule  variance  measured  in  dollars  implies  a  behind  schedule 
condition,  and  by  itself  is  a  good  quantitative  indicator  of  schedule  problem  severity,  it  is 
also  useful  to  view  schedule  problem  severity  in  terms  of  time.  The  method  used  in  this 
research  to  calculate  schedule  variance  in  terms  of  time  is  as  follows.  Because  the  current 
month  BCWS  represents  the  total  progress  scheduled  to  be  made  during  the  month,  it  can 
be  said  to  represent  one  month  worth  of  progress.  Likewise,  a  negative  current  month 
schedule  variance,  which  represents  the  amount  of  progress  scheduled  but  not  made 
during  the  month,  can  be  said  to  represent  between  zero  and  one  month  worth  of  not 
making  progress.  Thus,  when  the  negative  schedule  variance  is  compared  to  the  current 
month  BCWS,  the  resulting  ratio  (variance/BCWS)  can  be  said  to  represent  the  fraction  of 
a  month  in  which  progress  is  not  being  made  (AFSCP  173-4,  1989:  Para.  12- 1(d)).  This 
fraction  of  a  month  is  what  this  research  uses  to  represent  schedule  variance  in  terms  of 
time. 

Both  measures  of  schedule  variance  (dollars  and  time)  provide  information  useful 
in  quantifying  the  magnitude  of  schedule  problems.  Schedule  variance  measured  in  dollars 
tends  to  characterize  schedule  problems  on  large  efforts  as  much  greater  than  problems  on 
smaller  efforts.  This  is  because  larger  efforts  tend  to  spend  more  money  per  month  than 
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smaller  efforts.  A  month  of  delay  on  a  large  effort  may  cause  a  $1M  schedule  variance, 
while  a  month  of  delay  on  a  smaller  effort  may  only  cause  a  $5  OK  schedule  variance. 

Even  though  both  tasks  are  a  month  behind,  the  task  on  the  larger  effort  appears  to  have  a 
much  more  severe  schedule  problem.  This  is  consistent  with  the  view  that  a  delay  to  an 
effort  such  as  the  F-22  is  far  more  significant  than  a  delay  to  a  smaller  effort,  such  as  a 
tactical  communication  system. 

On  the  other  hand,  schedule  variance  measured  in  time  views  all  task  delays  as 
equal,  regardless  of  the  size  of  the  development  effort.  Thus,  even  though  the  schedule 
variance  (in  dollars)  on  a  large  effort  may  be  20  times  the  size  of  the  schedule  variance  (in 
dollars)  on  a  smaller  effort,  the  schedule  variance  (in  time)  for  both  cases  may  be  the  same. 
This  is  consistent  with  the  view  that  all  development  efforts  are  equally  important  and  that 
a  delay  to  the  F-22  is  no  more  or  less  significant  than  a  delay  to  a  smaller  tactical 
communication  system  effort.  Depending  on  the  reader’s  purpose  and  perspective,  both 
this  view  and  the  opposing  view,  as  described  in  the  preceding  paragraph,  may  be  equally 
valid.  For  this  reason,  this  research  collects  and  presents  both  measures  of  schedule 
problem  severity. 

One  final  note  on  the  use  of  schedule  variance  in  this  research.  Although  negative 
schedule  variances,  especially  those  stated  in  terms  of  time,  may  seem  to  imply  that  the 
overall  development  effort  is  experiencing  a  delay  equal  to  the  variance,  this  is  usually  not 
the  case.  In  fact,  only  when  tasks  are  on  the  critical  path  of  a  development  effort  do  their 
schedule  variances  indicate  delays  to  the  overall  effort  (Nicholas,  1990:284).  When  tasks 
are  not  on  the  critical  path,  task  delays  indicated  by  negative  schedule  variances  will 
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generally  be  absorbed  by  “slack"in  the  overall  develops,  effort's  schedule  so  asnor  ,o 
cause  a  delay  iu  the  overall  effort.  However,  even  if  negative  schedule  variances  ofleu  do 
no.  describe  a  delay  in  the  development  effort’s  completion  date,  they  are  still  a  valid 
measure  of  schedule  inefficiency,  and  therefore  pertain  directly  tins  research.  Further, 
the  fact  that  an  observed  schedule  problem  is  no,  associated  whh  a  task  on  the  cntical  path 
of  an  effort  does  no,  mean  tha,  a  similar  problem  will  not  occur  on  a  critical  path  task 


the  future. 


The  Sampling  Frame 


Tie  following  criteria  were  used  to  select  rite  system  development  efforts  tha,  are 
included  in  this  research.  First,  for  ease  of  access,  only  system  development  efforts  whose 
CPRs  were  available  in  tite  ASC  Cos,  Libras  were  considered.  This  includes  most  large 
Air  Force  system  development  efforts  managed  by  program  offices  a,  ASC. 

Second,  only  efforts  with  a  targe,  price  of  over  $40M  were  considered.  This 
selection  criteria  originates  ftom  tite  6c,  tha,  in  general,  CPRs  are  applied  to  larger 
efforts,  and  reports  known  as  Cos,  Schedule  Status  Reports  (C/SSRs,  are  applied  to 
sn^er  efforts  (AFSCP  ,73-4,  ,989:  Para.  3-4(a),.  As  explained  preriously,  CPRs  are 
the  sole  source  of  data  for  tins  research.  C/SSRs  were  no,  used  because  although  they 
contain  similar  information,  they  require  only  cumulative,  rather  titan  curten,  month 
reporting.  Also,  unlike  the  CPR,  where  BCWS  and  BCWP  for  a  task  must  be  calculated 
by  directly  manning  subtask  BCWS  and  BCWP,  C/SSRs  do  no,  have  a  standard  approach 
for  calculating  these  parameters  (AFSCP  173-4,  ,989:  Para.  3-4(b)).  Because  these 
parameters  are  used  to  calculate  schedule  variance,  variances  taken  from  CPRs  should  be 
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more  able  to  be  consistently  compared  across  development  efforts  than  variances  taken 
from  C/SSRs.  Also  aiding  in  consistency  is  the  fact  that  C/SCSC  (Cost/Schedule  Control 
Systems  Criteria)  is  required  on  contracts  using  the  CPR,  but  is  not  necessarily  applied  to 
contracts  using  the  C/SSR  (The  AFSC  Cost  Estimating  Handbook:  Para.  5.4.2).  Because 
contracts  applying  C/SCSC  have  accounting  and  reporting  systems  that  meet  the  same 
criteria,  cost  and  schedule  data  from  these  contracts  are  more  consistent  than  among 
contracts  not  applying  C/SCSC.  In  any  case,  limiting  the  sampling  frame  to  efforts  with  a 
target  price  of  over  $40M  also  controls  for  the  fact  that  small  efforts  may  have  different 
reasons  for  schedule  problems  than  large  efforts.  Eliminating  this  potential  moderating 
variable  (small  versus  large  efforts)  increases  the  credibility  of  the  results  of  this  research. 

Third,  this  research  only  considers  Engineering  and  Manufacturing  Development 
(EMD),  previously  known  as  Full  Scale  Development  (FSD),  efforts.  The  reason  for  this 
criteria  is  to  exclude  basic  research  and  exploratory  development  efforts,  which  tend  to  be 
neither  planned  nor  managed  with  the  same  emphasis  on  schedule  as  EMD  efforts. 

Fourth,  this  research  only  considers  development  efforts  that  are  ongoing,  or  that 
have  ended  after  1984.  This  timeframe  is  based  on  a  compromise  between  obtaining  a 
wide  variety  of  data  on  a  number  of  types  of  development  efforts,  and  ensuring  the  results 
are  relevant  to  current  and  future  efforts.  With  a  large  timeframe,  more  efforts  are 
included,  providing  a  wider  variety  of  data.  With  a  smaller  but  more  recent  timeframe, 
fewer  efforts  are  included,  however  the  schedule  problems  observed  are  more  likely  to 
represent  those  encountered  on  efforts  operating  under  today’s  management  practices. 
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Based  on  the  above  criteria,  the  ASC  Cost  Library  document  catalog  indicated  39 
system  development  efforts  described  by  1850  CPRs  would  comprise  the  sampling  frame. 
After  an  examination  of  several  CPRs  from  each  of  these  efforts,  five  development  efforts 
were  removed  for  not  containing  any  negative  schedule  variances,  four  were  removed  for 
not  reporting  explanations  for  schedule  variances,  three  were  removed  because  they  were 
not  currently  available,  two  were  removed  for  not  reporting  current  month  variances,  one 
was  removed  for  a  lack  of  data  due  to  a  late  contract  defimtization,  one  was  removed  for 
not  presenting  data  in  a  format  conducive  to  identifying  reasons  for  schedule  problems, 
and  one  was  removed  for  reporting  variances  against  functional  areas  rather  than  by  WBS 
elements.  In  addition,  the  number  of  usable  CPRs  on  four  efforts  was  less  than  anticipated 
because  reports  early  or  late  in  the  efforts  lacked  variance  explanations.  Table  3-1 
displays  the  final  sampling  frame  of  22  system  development  efforts  described  by  973 
CPRs. 


TABLE  3-1:  Description  of  Sampling  Frame 


Number  of 

Specific 

Years 

Number 

Contract 

Effort  Type 

Efforts 

Effort  Types 

Covered 

of  CPRs 

Value 

Aircraft/Missile 

7 

Fighters  (3) 

Bomber  (1) 
Transport  (1) 
Cruise  Missile  (1) 
Trainer  (1) 

1982-  1994 

316 

$  15,277  M 

Aircraft  Upgrade 

6 

Fighters  (2) 

Elec  Warfare  (2) 
Bomber  (1) 

Special  Msn  (1) 

1984-  1994 

190 

$  819  M 

Aircraft  Equipment 

5 

Recon  (2) 

Engine  (1) 
Launcher  (1) 
Transponder  (1) 

1981  -  1991 

290 

$  580  M 

Simulator 

4 

Elec  Warfare  (2) 
Aircrew  (2) 

1984-  1994 

177 

$  358  M 
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Note  that  in  Table  3-1  ,  the  actual  system  development  effort  (program)  names 
have  been  replaced  with  generic  program  types,  and  data  regarding  the  development  effort 
duration  and  cost  have  been  aggregated  in  order  to  allow  unrestricted  dissemination  of  this 
research.  By  omitting  clues  that  tie  the  data  to  specific  defense  programs,  sensitive 
information  is  properly  safeguarded,  yet  the  ability  to  fully  understand  and  appreciate  this 
research  is  preserved. 

The  Sampling  Process 

Based  on  the  previously  mentioned  C-17  pilot  study,  a  census  of  all  973  CPRs  in 
the  sampling  frame  would  be  too  time  consuming  for  a  limited  scope  research  effort  such 
as  this.  In  addition,  because  of  the  wide  variation  in  the  number  of  CPRs  associated  with 
the  efforts  in  the  sampling  frame,  a  census  would  represent  efforts  unequally.  In  order  to 
satisfy  the  research  objective  of  demonstrating  that  the  reasons  for  schedule  problems  on 
large  defense  system  development  efforts  are  common  across  efforts,  it  is  important  to 
ensure  efforts  are  represented  equally.  For  these  reasons,  a  sampling  approach  randomly 
selecting  an  equal  number  of  CPRs  from  each  effort  in  the  sampling  frame  for  examination 
was  adopted. 

Specifically,  nine  CPRs  were  randomly  selected  from  each  sampling  frame  system 
development  effort,  using  random  number  tables  (Kendall  and  Smith,  1938: 147-166)  to 
choose  among  the  CPRs  available  for  each  effort.  The  reason  for  selecting  nine  CPRs  per 
development  effort  is  that  the  effort  with  the  fewest  CPRs  had  only  nine  CPRs  associated 
with  it.  Thus,  nine  was  the  maximum  number  of  CPRs  that  could  he  examined  per  effort 
given  the  intent  was  to  examine  an  equal  number  of  CPRs  per  effort.  In  addition,  the 
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resulting  number  of  CPRs  (198)  was  deemed,  based  on  the  C-17  pilot  study,  to  represent 
an  ambitious  yet  reasonable  data  collection  and  analysis  effort  given  the  scope  of  this 
research.  The  exception  to  the  rule  of  examining  nine  CPRs  per  effort  occurred  towards 
the  end  of  the  data  collection  effort,  when  only  eight  reports  in  a  usable  format  were 
available  on  one  of  the  fighter  aircraft  development  efforts.  In  this  case,  only  eight  CPRs 
were  examined,  however  there  is  no  evidence  that  this  discrepancy  significantly  affects  the 
results  of  this  research. 

The  Data  Collection  Process 

Data  collection  for  this  research  was  a  two-step  process.  In  the  first  step,  relevant 
data  was  copied  from  all  sampled  CPRs.  Specifically,  for  each  sampled  CPR,  every  page 
containing  either  schedule  variance  or  narrative  related  to  an  explained  current  month 
negative  schedule  variance  reported  against  WBS  elements  (contained  in  Format  V  of  the 
CPR)  was  copied  either  with  a  standard  office  or  microfiche  copier.  In  addition.  Format  I 
of  each  CPR,  which  contains  BCWS  values  for  the  WBS  elements,  was  copied  in  the  same 
manner. 

In  the  second  step,  the  CPR  pages  copied  in  step  one  were  examined  such  that 
each  explained  current  month  negative  schedule  variance  reported  against  a  WBS  element 
generated  an  entry  in  a  computer  database.  For  each  entry,  the  program  name,  reason  for 
the  schedule  variance  (extracted  from  the  narrative),  schedule  variance,  BCWS,  and  CPR 
date  were  recorded.  In  recording  this  information,  the  following  guidelines  were 
followed.  First,  only  negative  schedule  variances  that  represented  a  true  behind  schedule 
condition  for  a  task  generated  a  data  entry.  Variances  resulting  from  accounting  errors  or 
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late  billing  for  completed  work  were  not  included.  Similarly,  false  negative  variances 
resulting  from  work  completed  early  or  out  of  planned  sequence  were  not  included.  An 
example  of  how  a  task  actually  completed  early  can  generate  a  false  negative  schedule 
variance  is  as  follows.  Suppose  a  task  is  scheduled  to  occur  in  the  month  of  May,  but 
instead  occurs  in  April.  Because  no  effort  was  scheduled  for  April,  the  actual  progress 
made  during  April  will  cause  a  positive  schedule  variance  (as  expected).  However, 
because  the  effort  was  originally  scheduled  for  May,  and  no  progress  occurs  in  May 
(because  the  effort  is  complete),  a  false  negative  schedule  variance  is  reported.  If  only  the 
May  report  is  sampled,  the  negative  variance  will  appear  to  indicate  a  behind  schedule 
condition  when  in  fact  it  does  not.  For  this  reason,  data  associated  with  such  false 
variances  has  been  excluded  from  this  research. 

Second,  the  narratives  associated  with  explained  negative  current  month  schedule 
variances  were  often  written  in  paragraph  form.  In  order  to  facilitate  data  collection  and 
analysis,  these  paragraphs  were  summarized  in  one-line  descriptions  of  75  characters  or 
less.  Where  possible,  explanations  were  recorded  verbatim,  although  most  explanations 
were  paraphrased.  In  summarizing  the  explanation,  the  primary  focus  was  on  preserving 
the  actual  reason  for  the  schedule  variance,  while  a  secondary  emphasis  was  on  capturing 
how  the  schedule  problem  impacted  the  development  effort. 

Third,  in  many  cases,  the  explanation  associated  with  a  single  schedule  variance 
reports  multiple  reasons  for  that  variance.  Although  this  occurs  more  often  in 
explanations  of  cumulative  variances  and  variances  reported  against  functional  areas,  it 
occurs  often  enough  in  explanations  of  current  month  schedule  variances  reported  against 
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WBS  elements  that  the  following  rules  were  adopted  for  use  in  this  research.  First,  if 
more  than  one  reason  was  associated  with  a  single  schedule  variance,  a  data  entry  was 
created  for  each  reason.  Thus,  a  single  reported  schedule  variance  on  a  CPR  could  result 
in  multiple  entries  in  the  database  supporting  this  research.  This,  however,  created  a 
problem  in  deciding  how  to  divide  the  variance  among  its  associated  reasons,  hence  the 
second  set  of  rules.  On  occasion,  the  narrative  will  identify,  either  in  dollars  or  in  percent, 
the  amount  of  variance  associated  with  each  included  reason.  In  these  cases,  variance  was 
divided  accordingly.  More  often,  the  narrative  will  state  that  the  variance  was  caused 
primarily  by  one  reason,  but  other  reasons  contributed.  In  these  cases,  the  variance  was 
divided  in  a  ratio  of  3  to  1  if  there  were  two  reasons,  and  3  to  1  to  1  if  there  were  three 
reasons.  Most  often,  the  narrative  contains  no  indication  of  one  reason  being  any  more 
significant  than  another.  In  these  cases,  variance  was  divided  equally  among  each  reason. 

The  final  rule  for  dividing  quantitative  information  among  reasons  for  schedule 
problems  is  that  when  multiple  reasons  are  associated  with  a  single  variance,  and  hence  a 
single  BCWS,  on  a  CPR,  the  BCWS  is  not  divided  among  the  reasons.  Instead,  the  total 
BCWS  associated  with  the  reported  variance  is  applied  to  each  recorded  reason.  At  first, 
there  may  appear  to  be  an  inconsistency  between  dividing  variance  and  not  dividing 
BCWS.  The  reason  for  the  apparent  discrepancy  is  that  while  variance  is  fully  described 
by  the  reasons  listed  in  the  narrative,  BCWS  is  based  on  the  completion  of  tasks  that  may 
or  may  not  be  listed  in  the  narrative.  Recall  that  BCWS  reflects  all  work  that  is  scheduled 
to  occur  on  a  given  WBS  element,  and  schedule  variance  only  reflects  work  not 
accomplished.  When  variance  is  divided,  it  merely  is  an  attempt  to  distribute  a  known 
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amount  of  inefficiency,  relative  to  the  completion  of  a  WBS  element,  among  known 
reasons  for  that  inefficiency.  If  BCWS  were  to  be  divided  in  the  same  manner,  it  would 
imply  that  each  reason  for  schedule  variance  was  creating  a  given  amount  of  inefficiency 
against  its  very  own  lower  level  WBS  element.  While  this  is  possible,  it  certainly  is  not 
expected.  Given  the  absence  of  information  on  how  multiple  reasons  associated  with  a 
single  reported  schedule  variance  impact  lower  level  WBS  elements,  this  thesis  has  applied 
the  full  BCWS  associated  with  each  reported  variance  to  each  reason  associated  with  that 
variance. 

Method  of  Data  Analysis 

It  is  necessary  to  group  data  associated  with  similar  reasons  into  categories  in 
order  to  analyze  the  collected  data  in  a  manner  that  allows  meaningful  identification  of  the 
reasons  for  schedule  problems  across  system  development  efforts,  quantification  of  the 
severity  of  the  problems  associated  with  those  reasons,  and  demonstration  that  those 
reasons  are  common  across  efforts.  Because  this  research  is  descriptive  in  nature,  no 
attempt  was  made  to  force  the  data  into  pre-defined  categories.  Instead,  categories  were 
developed  to  reflect  the  data.  A  discussion  of  the  method  for  categorizing  the  data  is  as 
follows. 

The  initial  categorization  of  the  data  into  groups  having  similar  reasons  for 
schedule  problems  was  conducted  in  parallel  with  the  entry  of  CPR  data  into  the  computer 
database.  As  each  reason  for  schedule  problems  and  associated  quantitative  information 
was  extracted  from  the  CPR,  the  reason  was  categorized  based  on  its  wording  and  the 
researcher’s  five  years  of  experience  as  an  Air  Force  project  manager.  This  categorization 
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was  intended  to  represent  the  reason  at  a  high  enough  level  of  generality  to  allow  the 
reduction  of  hundreds  of  data  entries  into  a  manageable  and  understandable  form,  while 
preserving  the  descriptive  nature  of  the  original  reason.  For  each  data  entry,  if  the  reason 
for  the  schedule  problem  was  similar  to  an  existing  category,  the  reason  was  placed  in  that 
category.  In  the  event  that  a  reason  did  not  fit  into  any  categories,  a  new  category  was 
defined  based  on  the  above  method. 

Once  all  the  data  was  entered  into  the  computer  database  and  grouped  according 
to  initial  categories,  the  researcher  conducted  two  separate  reviews  to  ensure  appropriate 
categorization  of  the  data.  The  first  review  was  to  ensure  the  reasons  for  schedule 
problems  truly  belonged  in  their  assigned  category.  This  review  also  was  to  ensure  the 
categories  themselves  were  at  similar  levels  of  detail.  Based  on  this  review,  several 
reasons  were  recategorized,  and  several  categories  at  too  low  a  level  of  detail  were 
divided  into  more  detailed  categories.  As  an  example  of  the  latter,  a  category  named 
“development  problems”  was  divided  into  “technical  problems,”  “technical  definition,”  and 
“manufacturing  problems.” 

The  second  review  was  conducted  primarily  to  ensure  that  the  categories  focused 
on  the  reason  for  the  schedule  problem,  and  not  what  the  problem  impacted  (such  as 
integration),  or  where  the  problem  occurred  (such  as  at  the  subcontractor’s  facility  versus 
prime  contractor’s  facility).  Based  on  the  review,  several  categories  were  eliminated,  and 
their  reasons  were  assigned  to  other  categories.  Following  this  reassignment,  the  review 
reexamined  whether  each  reason  belonged  in  its  assigned  category.  This  time,  only  four 
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reasons  were  recategorized,  demonstrating  that  the  categories  were  now  stable  enough  to 
allow  meaningful  analysis. 

Following  the  grouping  of  reasons  for  schedule  problems  on  large  system 
development  efforts  into  descriptive  categories,  the  reasons  within  each  category  were 
further  grouped  into  subcategories.  These  descriptive  subcategories  were  determined  in 
the  same  manner  as  the  categories.  Their  purpose  is  to  provide  a  level  of  detail  between 
the  fairly  general  categories  and  the  detailed  data  entries.  This  additional  detail  allows 
greater  insight  into  the  reasons  for  schedule  problems  by  identifying  the  types  of  reasons 
that  make  up  each  category.  In  addition,  the  subcategories  allow  for  a  more  orderly 
grouping  of  data  entries  in  the  full  listing  of  the  computer  database  in  the  appendix. 

After  fully  categorizing  and  subcategorizing  the  data,  the  data  was  analyzed  by 
category  in  terms  of  the  previously  discussed  schedule  problem  severity  measures  of 
frequency,  schedule  variance  (in  dollars),  and  schedule  variance  (in  time).  The  data  was 
also  analyzed  to  demonstrate  that  the  reasons  for  schedule  problems  are  common  across 
system  development  efforts.  The  details  of  these  analyses  are  presented  in  Chapter  4. 
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I  V:  Data  Description  and  Analysis 


Introduction 

This  research  had  three  mam  objectives.  The  first  was  to  identify  the  actual 
reasons  for  schedule  problems  across  large  Air  Force  system  development  efforts, 
describing  those  reasons  at  a  level  of  detail  that  will  allow  the  development  of  appropriate 
corrective  actions.  The  second  was  to  quantify  the  importance  of  each  category  of 
reasons,  in  terms  of  frequency  and  severity,  in  order  to  determine  the  categories  of  reasons 
most  and  least  deserving  of  management  attention.  The  third  was  to  demonstrate  that  the 
reasons  for  schedule  problems  are  not  program  unique,  but  are  common  across  system 
development  efforts,  therefore  schedule-related  lessons  learned  from  past  and  present 
efforts  are  likely  to  be  relevant  to  future  efforts.  The  following  chapter  presents  a 
characterization  of  the  data  collected  in  this  research,  and  an  analysis  of  that  data  with  the 

intent  of  satisfying  the  above  research  objectives. 

Specifically,  it  discusses  the  number  of  schedule  problem  observations  obtained 
from  each  type  of  development  effort,  the  number  of  observations  per  development  effort, 
the  number  of  observations  by  year  of  occurrence,  the  categories  into  which  the  reasons 
for  schedule  problems  were  grouped,  and  the  types  of  reasons  composing  each  category. 
The  chapter  then  compares  the  categories  of  reasons  for  schedule  problems  in  terms  of 
frequency,  total  schedule  variance  per  category  (in  both  dollars  and  work  days),  and 
average  schedule  variance  per  category  (in  both  dollars  and  work  days).  The  chapter 
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concludes  with  a  summary  ranking  of  each  category  in  terms  of  the  above  measures,  and 
an  analysis  demonstrating  that  the  reasons  for  schedule  problems  are  common  across 
system  development  efforts. 

General  Description  of  Data 

Using  the  methodology  addressed  in  Chapter  3,  549  instances  of  reasons  for 
schedule  problems  across  22  system  development  efforts  were  observed.  In  order  to 
provide  a  context  within  which  to  assess  the  results  of  this  research,  the  following  section 
broadly  characterizes  these  reasons  in  terms  of  where  and  when  the  reasons  occurred. 

Figure  4-1  provides  a  graphical  representation  of  the  proportion  of  reasons  for 
schedule  problems  observed  on  each  of  the  types  of  system  development  efforts 
(programs)  defined  in  Chapter  3,  Table  3-1. 
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9%  A/C 


FIGURE  4-1:  Proportion  of  Reasons  from  Each  Development  Effort  Type 
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Although  Figure  4- 1  may  appear  to  imply  that  schedule  problems  are  more  likely 
to  occur  on  one  type  of  development  effort  than  another,  this  research  neither  supports 
nor  refutes  that  conclusion.  As  explained  in  Table  3-1,  the  number  of  efforts,  size  of 
efforts,  and  length  of  efforts  is  not  consistent  across  development  effort  types.  For  this 
reason,  comparisons  among  the  effort  types  in  this  research  is  not  appropriate.  Instead, 
the  utility  of  Figure  4-1  is  that  it  assists  in  understanding  the  origin  of  the  data  underlying 
this  research  so  that  results  of  this  research  can  be  correctly  interpreted. 

To  further  assist  in  understanding  the  origin  of  the  data,  Figure  4-2  presents  the 
number  of  observed  reasons  for  schedule  problems  per  sampled  development  effort. 


FIGURE  4-2:  Reasons  Observed  per  Development  Effort 
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From  Figure  4-2,  it  is  clear  that  not  all  development  efforts  provided  the  same 
number  of  reasons  for  schedule  problems.  This  is  despite  the  fact  that,  as  explained  in 
Chapter  3,  the  data  was  collected  from  an  identical  number  of  monthly  Cost  Performance 
Reports  (CPRs)  for  each  development  effort.  Although  the  cause  of  this  variation  was  not 
investigated  in  this  research,  potential  reasons  could  include  varying  schedule  problem 
reporting  thresholds  among  programs  and  varying  schedule-related  success  among 
programs.  In  any  case,  for  the  purposes  of  this  research,  it  is  important  to  note  that 
certain  development  efforts  influence  the  results  more  than  others,  but  that  most  efforts 
provided  a  reasonable  contribution.  Overall,  the  average  number  of  reasons  for  schedule 
problems  observed  per  effort  was  roughly  25,  with  a  standard  deviation  also  of  roughly 
25. 

Finally,  Figure  4-3  presents  the  number  of  reasons  for  schedule  problems  observed 
in  all  CPRs  sampled  in  a  given  year. 

f - - - -  ! 
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FIGURE  4-3:  Reasons  Observed  by  Year  of  Occurrence 
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Although  Figure  4-3  may  appear  to  imply  that  schedule  problems  are  more  likely 
to  occur  on  recent  efforts  than  on  efforts  prior  to  1989,  this  research  neither  supports  nor 
refutes  that  conclusion.  In  the  research,  there  was  no  attempt  to  sample  data  equally 
across  years,  and  as  such,  conclusions  regarding  the  prevalence  of  schedule  problems  by 
year  are  not  appropriate.  Instead,  the  utility  of  Figure  4-3  is  that  it  assists  in  understanding 
the  origin  of  the  data  underlying  this  research  in  order  that  results  of  this  research  can  be 
correctly  interpreted.  For  example,  based  on  Figure  4-3,  it  is  clear  that  a  significant 
proportion  of  the  data  underlying  this  research  reflects  recent  schedule  problems 
(experienced  since  1989),  which  should  provide  managers  with  added  confidence  that  the 
results  of  this  research  will  apply  well  to  current  efforts,  and  to  efforts  in  the  near  future. 
Categorization  of  Reasons  for  Schedule  Problems 

In  order  to  make  sense  of  the  549  reasons  for  schedule  problems  observed  in  this 
research,  these  reasons  have  been  grouped  into  descriptive  categories  according  to  the 
methodology  specified  in  Chapter  3.  Because  these  categories  are  intended  to  reflect  the 
descriptive  nature  of  the  original  reasons,  albeit  at  a  lower  level  of  detail,  the  best  way  to 
describe  these  categories  is  through  examples  of  the  actual  reasons  they  summarize.  In 
this  manner,  Table  4-1  provides  a  brief  description  of  each  of  the  20  categories  used  in  this 
research. 
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TABLE  4-1:  Categories  of  Reasons  for  Schedule  Problems 


Category 

Examples 

Contracting 

Contract  not  yet  signed  with  vendor  for  wing  slat  package 

Late  source  selections  delaying  H/W  &  S/W  design 

Changed  Plans 

Minor  changes  in  several  functional  responsibilities 

Rescheduled  design  effort 

Design  Changes 

Design  changes  due  to  weight  reduction  activities 

Tech  manuals  delayed  due  to  frequently  changing  tech  data 

Estimating 

Number  of  detail  parts  to  be  custom  designed  more  than  planned 

Overly  aggressive  material  plan  could  not  be  met 

Facility  Problems 

Facility  design  behind  schedule 

Design  mods/layouts  behind  due  to  late  test  area  completion 

Gov't  Added  Work 

Gov't  directed  more  detailed  specifications  than  anticipated 

Support  for  VIP  demo  flights  impacting  test  effort 

Gov't  Not  Supportive 

Delay  in  obtaining  source  data  from  Gov't  for  tech  manuals 

Late  final  test  plan  due  to  delayed  customer  comments 

Gov't  Stopped  Work 

Gov't  directed  work  stoppage  impacting  other  areas 

Stop  work  order  slowed  vendor  tasks 

Inventory  Mgt 

Delays  in  recognition  of  receipt  of  vendor  deliveries 

Parts  shortages  impacting  test  article  fabrication 

Late  Data 

Lack  of  imagery  data  to  validate  algorithms 

Late  engineering  drawing  releases 

Late  Reviews 

Delayed  fabrication  due  to  slip  of  CDR 

Additional  effort  required  to  close  out  CDR 

Miscellaneous  Delays 

Delay  in  shipment  overseas  impacts  test 

Site  activation  meeting  delayed 

Manufacturing  Probs 

Delays  in  fabrication  of  major  assembly  tooling  fixtures 

Delay  in  defining/releasing  production  bill  of  materials 

Quality 

Req’ments  Changes 

Vendor  testing  not  satisfactory  for  acceptance 

Flight  station  shipped  in  an  incomplete  condition  by  subktr 
Changing  specifications  impacting  effort 

Change  in  harness  board  requirements 

Staffing 

Inadequate  systems  engineering  staffing  delaying  specifications 

Reassignment  of  personnel  to  more  critical  areas 

Subcontractor  Late 

Test  Problems 

Late  material  deliveries  delaying  STE  effort 

Subcontract  or  delay  s  in  reaching  development  milestones 
Unplanned  instrumentation  modification  delaying  flight  test 

Component  difficulties  during  qual  testing 

Technical  Definition 

Delay  in  specification  generation 

Incorrect  envelope  definition  to  sub  delayed  CDR 

Technical  Problems 

Difficulties  in  analyses  and  monte  carlo  simulation 

Technical  difficulties  associated  with  H/W  algorithms 
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The  categories  listed  in  Table  4- 1  identify  the  reasons  for  schedule  problems  across 
Air  Force  system  development  efforts  at  fairly  general  level.  It  is  at  this  level  that 
comparisons  of  reasons  in  terms  of  frequency  of  observation  and  severity  of  associated 
problems  have  been  made.  These  categories  have  also  been  used  to  demonstrate  the 
commonality  of  reasons  across  development  efforts.  Prior  to  making  these  comparisons 
and  demonstrations,  however,  it  is  essential  that  the  above  categories  are  fully  understood. 
In  the  following  subsections,  each  category  listed  in  Table  4-1  is  described  based  on 
subcategories  (as  explained  in  Chapter  3)  of  reasons  assigned  to  each  category.  By 
understanding  the  categories  in  these  more  detailed  terms,  the  likelihood  of  formulating 
appropriate  corrective  actions  for  schedule  problems  based  on  the  results  of  this  research 
is  increased. 

“Contracting”  Category.  This  category  includes  reasons  related  to  contractual 
actions  and  the  process  of  awarding  subcontracts.  In  the  assignment  of  reasons  to  this 
category,  most  did  not  contain  sufficient  information  to  determine  whether  or  not  the 
associated  schedule  problems  were  caused  by  contracting  functional  procedures  and  staff. 
Consequently,  this  category  includes  contracting-related  reasons  for  schedule  problems 
that  were  likely  caused  not  only  by  the  contracting  department,  but  by  other  departments 
failing  to  provide  information  or  generate  requests  in  a  timely  manner,  and  by  management 
actions  resulting  in  lengthy  contracting-related  approval  cycles.  Specific  subcategories  of 
reasons  for  schedule  problems  within  the  “contracting”  category  are  as  follows: 

(1)  source  selection  of  subcontractors  and  vendors  (including  request  for  proposal 
(RFP)  preparation  and  proposal  receipt), 


40 


(2)  placing  subcontractors  and  vendors  on  contract, 

(3)  processing  purchase  orders,  and 

(4)  terminating  subcontractors. 

Figure  4-4  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  “contracting”  category. 


FIGURE  4-4:  “Contracting”  Subcategories 


“Changed  Plans”  Category.  This  category  includes  reasons  related  to  revised 
schedules  and  work  plans.  Specific  subcategories  of  reasons  for  schedule  problems  within 
the  “changed  plans”  category  are  as  follows: 

(1)  new  design  schedules, 

(2)  new  delivery  schedules,  and 

(3)  changes  in  work  responsibilities  among  functional  departments. 
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Figure  4-5  presents  the  relative  contribution  of  reasons  (in  terms  of  number 


observed)  within  the  above  subcategories  to  the  “changed  plans”  category. 


FIGURE  4-5:  “Changed  Plans”  Subcategories 


“Design  Changes”  Category.  This  category  includes  reasons  related  to  changes 
in  system  or  component  designs,  typically  undertaken  to  fix  problems  or  to  improve 
performance.  Quite  often,  in  addition  to  requiring  time  to  implement,  these  changes  also 
impact  other,  related  tasks  that  depend  on  stable,  defined  designs  for  their  continued 
progress.  Specific  subcategories  of  reasons  for  schedule  problems  within  the  “design 
changes”  category  are  as  follows: 
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(1)  changes  affecting  generation  of  data,  such  as  that  needed  for  engineering 

drawings  or  technical  manual  preparation; 

(2)  changes  affecting  manufacturing  activities,  such  as  mockup  construction, 

component  fabrication,  and  system  assembly; 

(3)  changes  affecting  subcontractor  deliveries  (deliveries  delayed  due  to  the  need 

to  incorporate  changes); 

(4)  changes  affecting  testing  (tests  that  must  wait  for  redesigned  components); 

(5)  general  delay,  including  changes  limited  to  affecting  design  efforts,  or  changes 

whose  affect  is  unspecified;  and, 

(6)  changes  to  reduce  the  weight  of  the  system 

Figure  4-6  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  “design  changes”  category. 
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“Estimating”  Category.  This  category  includes  reasons  related  to  the 
underestimation  of  time  or  effort  required  to  complete  tasks  on  schedule.  Although  many 
of  the  549  observed  reasons  for  schedule  problems  may  indeed  have  been  caused  by  poor 
schedule  estimating,  the  reasons  included  in  this  category  are  only  those  that  mention, 
either  directly  or  indirectly,  a  problem  in  schedule  planning  or  estimating.  Specific 
subcategories  of  reasons  for  schedule  problems  within  the  “estimating”  category  are  as 
follows: 

(1)  overly  ambitious  schedules, 

(2)  optimistic  material  budgets  (based  either  on  using  more  material  during  the 

observed  month  because  the  effort  was  planned  to  be  further  towards 
completion,  or  on  overly  optimistic  supply  predictions), 

(3)  planning  to  an  incorrect  schedule, 

(4)  underestimating  the  time  required  to  order  materials, 

(5)  lack  of  integrated  schedules  resulting  in  disconnects  among  tasks, 

(6)  misplanning  (poor  planning)  of  tasks, 

(7)  underestimating  the  time  needed  to  complete  tasks,  and 

(8)  underestimating  the  amount  of  work  required  to  complete  tasks. 

Figure  4-7  presents  the  relative  contribution  of  reasons  (in  terms  of  number 

observed)  within  the  above  subcategories  to  the  “estimating”  category. 
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FIGURE  4-7:  “Estimating”  Subcategories 


“Facility  Problems”  Category.  This  category  includes  reasons  related  to  the 
design,  fabrication,  and  renovation  of  facilities  required  for  development  effort 
completion.  Because  only  three  reasons  have  been  included  in  this  category,  two  of  which 
are  included  in  Table  4-1,  no  further  description  of  this  category  is  necessary. 

“Government  Added  Work”  Category.  This  category  includes  reasons  related 
to  government  direction  that  resulted  in  additional,  unplanned  effort  for  the  contractor. 
Specific  subcategories  of  reasons  for  schedule  problems  within  the  “government  added 
work”  category  are  as  follows: 

(1)  additional  unplanned  government  review  of  contractor  efforts, 

(2)  government  directed  changes  in  design  and  documentation, 

(3)  government  comments  at  reviews  generating  contractor  action  items. 
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(4)  government  requested  marketing  support  to  help  “sell”  the  development  effort 

to  higher-ups,  and 

(5)  government  rescheduling  of  reviews  causing  inefficiencies  in  the  work 

schedules  of  those  supporting  the  reviews. 

Figure  4-8  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  “government  added  work”  category. 


FIGURE  4-8:  “Government  Added  Work”  Subcategories 


“Government  Not  Supportive”  Category.  This  category  includes  reasons 
related  to  a  government  failure  to  provide  an  appropriate  level  of  support  to  the 
contractor.  Specific  subcategories  of  reasons  for  schedule  problems  within  the 
“government  not  supportive”  category  are  as  follows: 
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(1)  late  data  item  approval  (government  was  late  in  approving  submitted  data 

items), 

(2)  failure  of  the  government  to  provide  required  data  to  the  contractor  (includes 

classified  and  unclassified  operational,  threat,  and  design-related  data  that 
requires  government  collection  or  release), 

(3)  government  funding  shortfalls  impacting  contractor  task  completion, 

(4)  incomplete  or  late  government  furnished  equipment  or  property,  and 

(5)  late  direction  from  the  government  (including  comments,  approvals,  inputs,  and 

decisions). 

Figure  4-9  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  “government  not  supportive”  category. 
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FIGURE  4-9:  “Government  Not  Supportive”  Subcategories 
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“Government  Stopped  Work-  Category,  This  categoty  mcludes  reasons  ^ 
to  a  government-directed  stop  wotk  order.  Reasons  in  to  category  reflect  schedule 
variances  both  in  stopped  tasks,  and  in  tasks  related  to  the  stopped  tasks.  Because  Are 
reasons  included  in  this  category  are  vety  similar.  Table  4-1  is  sufficiently  descriptive  of 
this  category  that  no  further  elaboration  is  required. 

“Inventory  Management-  Category.  This  categoty  includes  reasons  rektted  to 
inventory  problems.  Specific  subcategories  ofreasons  for  schedule  problems  wflhin  the 
inventory  management”  category  are  as  follows: 


(1)  ineffective  controls  for  materials  being  ordered,  delivered,  or  stored,  such  as 

being  unaware  of  delivered  material,  or  having  parts  in  stock  that  do  not 
match  the  bill  of  materials;  and, 

(2)  parts  shortages. 


Figure  4-10  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  ‘inventory  management”  category. 
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Late  Data  Category.  This  category  includes  reasons  related  to  the  late  receipt 
or  generation  of  required  information  either  within  the  contractor’s  organization,  or 
between  contractor  and  subcontractor.  Specific  subcategories  of  reasons  for  schedule 
problems  within  the  ‘late  data”  category  are  as  follows: 

(1)  incomplete  data  items  (late  completion  of  specifications  and  other  documents 

required  for  delivery  to  the  government); 

(2)  late  engineering  release  of  drawings  and  other  design  data  required  for 

manufacturing  and  other  activities;  and, 

(3)  late  or  incomplete  information  (such  as  specifications,  reports,  and  data) 

impacting  areas  such  as  design,  manufacturing,  training,  provisioning, 

technical  publications,  test,  facilities,  and  material  orders. 

Figure  4-11  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  ‘late  data”  category. 
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Review  Category.  This  category  includes  reasons  related  to  technical  or 
management  reviews  that  have  been  enter  lengthened  or  postponed  by  the  comment  or 

subcontractor.  Specific  snbeategories  of  reasons  for  schedule  problems  within  the  “late 
review”  category  are  as  follows: 

(1)  review  completion  (more  than  the  anticipated  amount  oftime  was  required  to 

conduct  the  review  and  address  issues  that  surfaced  during  the  review);  and 

(2)  late  start  (“slip”)  of  scheduled  requirements  or  design  reviews,  often  impacting 

activities  such  as  design,  fabrication,  and  test. 

Figure  4-12  presents  the  relative  contribution  of  reasons  (in  terms  ofnumber 
observed)  within  the  above  subcategories  to  the  ‘late  review”  category. 
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“Miscellaneous  Delays”  Category.  This  category  includes  schedule  variance 
explanations  that  failed  to  state  the  reason  for  a  particular  schedule  problem,  and  reasons 
that  did  not  fall  neatly  into  another  category.  The  explanations  include  various  schedule 
problems  whose  reason  was  unspecified,  impacting  activities  such  as  design,  logistics, 
deployment,  test,  and  manufacturing.  The  reasons  in  this  category  include  delays  in 
overseas  shipments  and  an  inefficient  management  process.  Because  of  the  diverse  nature 
of  the  reasons  and  explanations  in  this  category,  subcategorization  is  not  particularly 
useful  in  this  case,  and  as  such,  has  not  been  conducted. 

“Manufacturing  Problems”  Category.  This  category  includes  reasons  related  to 
problems  building  hardware,  both  in  the  development  and  preparation  for  production  of  a 
system.  These  problems,  described  below,  are  those  encountered  in  translating  an 
engineering  design  into  developmental  and  production  hardware.  Specific  subcategories 
of  reasons  for  schedule  problems  within  the  “manufacturing  problems”  category  are  as 
follows: 

(1)  miscellaneous  problems,  such  as  late  requisitioning  of  inventory,  a  slowdown 

due  to  a  new  computer  system,  and  design  problems  discovered  during 
fabrication; 

(2)  fabrication  problems  on  breadboards,  system  components,  tooling,  and  test 

equipment; 

(3)  late  receipt  of  material  impacting  fabrication  and  assembly; 

(4)  late  start  of  tooling; 
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(5)  manufacturing  design  problems  in  developing  the  bill  of  materials,  releasing 

tooling,  and  providing  articles  required  for  development; 

(6)  machine  proofing  problems  and  delays  impacting  fabrication  and  assembly;  and, 

(7)  manufacturing  process  problems,  such  as  a  fabrication  process  that  produces 

system  components  that  do  not  meet  the  specification. 

Figure  4-13  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  “manufacturing  problems”  category. 


FIGURE  4-13:  “Manufacturing  Problems”  Subcategories 


“Quality”  Category.  This  category  includes  reasons  related  to  either  a  lack  of 
quality  (in  terms  of  defective  equipment/components,  inadequate  testing,  and  substandard 
personnel  performance),  or  delays  caused  by  quality  assurance  activities.  Most  of  these 
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reasons  deal  with  subcontractor  or  vendor  problems.  Specific  subcategories  of  reasons  for 
schedule  problems  within  the  “quality”  category  are  as  follows: 

(1)  miscellaneous  problems,  such  as  poor  subcontractor  performance  or  inadequate 

preparation  for  testing; 

(2)  inadequate  testing,  such  as  when  vendor  in-house  testing  is  insufficient  to 

determine  whether  or  not  a  product  is  acceptable; 

(3)  unacceptable  items,  that  were  either  delivered  with  missing  components  or  did 

not  meet  specifications;  and, 

(4)  acceptance  procedures,  such  as  inspections  or  approvals  by  the  prime 

contractor,  delaying  subcontractor  deliveries. 

Figure  4-14  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  “quality”  category. 


FIGURE  4-14:  “Quality”  Subcategories 
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“Requirements  Changes”  Category.  This  category  includes  reasons  related  to 
changes  in  security,  test,  and  system  component  requirements,  as  reflected  in 
specifications  or  otherwise.  This  category  is  different  from  the  “design  changes  category 
in  that  design  changes  are  made  in  order  to  reach  an  objective,  whereas  requirements 
changes  alter  the  objective.  Although  more  of  the  549  reasons  for  schedule  problems  may 
indeed  belong  in  this  category,  only  those  reasons  in  which  changing  requirements  were 
either  directly  or  indirectly  mentioned  were  included.  Because  “requirements  changes 
adequately  captures  the  essence  of  all  reasons  included  in  this  category,  further 
subcategorization  is  of  little  value,  and  hence  was  not  conducted. 

“Staffing”  Category.  This  category  includes  reasons  related  to  having  insufficient 
personnel  assigned  to  tasks.  Specific  subcategories  of  reasons  for  schedule  problems 
within  the  “staffing”  category  are  as  follows: 

(1)  hiring  delays,  either  during  the  initial  ramp-up  of  personnel  to  conduct  the 

effort,  or  in  replacing  personnel  later  in  the  effort; 

(2)  inadequate  staffing  for  the  timely  completion  of  tasks  either  at  contractor  or 

subcontractor  facilities,  impacting  such  activities  as  design,  manufacturing, 

test,  and  technical  publications; 

(3)  receipt  of  security  clearances  taking  longer  than  expected,  causing  delays  until 

sufficient  numbers  of  cleared  personnel  are  available  to  work  on  the  effort; 

(4)  reassignment  of  personnel  to  higher  priority  or  nearer  term  tasks,  resulting  in 

delays  on  the  original  tasks;  and. 
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(5)  wrong  people  assigned  to  a  task,  such  as  when  subcontractors  with  either  a 
lack  of  required  expertise  or  with  an  inappropriate  mix  of  personnel  are 
working  on  a  task. 

Figure  4-15  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  “staffing”  category. 


FIGURE  4-15:  “Staffing”  Subcategories 


“Subcontractor/Vendor  Late”  Category.  This  category  includes  reasons  related 
to  late  deliveries  or  slow  progress  by  subcontractors  or  vendors.  Specific  subcategories  of 
reasons  for  schedule  problems  within  the  “subcontractor/vendor  late”  category  are  as 
follows: 
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(1)  late  deliveries  of  software,  hardware  (including  anything  from  connectors  to 

entire  radar  or  head-up  display  units),  and  other  miscellaneous  products 
impacting  prime  contractor  activities  such  as  design,  manufacturing,  and 
test; 

(2)  late  deliveries  by  a  subcontractor’s  subcontractor  (essentially  the  same  type  of 

reasons  as  in  (1)  above,  however  delivery  is  late  to  subcontractor  rather 
than  to  prime  contractor);  and 

(3)  slow  progress  of  a  subcontractor  toward  meeting  its  planned  schedule. 

Figure  4-16  presents  the  relative  contribution  of  reasons  (in  terms  of  number 

observed)  within  the  above  subcategories  to  the  “subcontractor/vendor  late”  category. 


FIGURE  4-16:  “Subcontractor /Vendor  Late”  Subcategories 

“Test  Problems”  Category.  This  category  includes  reasons  related  to  problems 
encountered  in  preparing  for  or  conducting  component  or  system  ground  or  flight  testing. 
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Included  reasons  are  associated  with  schedule  problems  in  simulation,  instrument 
modification,  and  test  asset  receipt,  as  well  as  those  encountered  in  environmental, 
transportation,  qualification,  component,  and  flight  testing.  Because  the  reasons  in  this 
category  are  adequately  described  as  “test  problems,”  and  because  the  varied  specifics  of 
each  reason  makes  grouping  below  the  category  level  extremely  difficult,  further 
subcategorization  was  not  conducted. 

“Technical  Definition”  Category.  This  category  includes  reasons  related  to 
defining  requirements,  interfaces,  designs,  and  tasks  to  a  level  where  work  can  proceed 
without  being  hampered  by  a  lack  of  appropriate  technical  direction.  Specific 
subcategories  of  reasons  for  schedule  problems  within  the  “technical  definition”  category 
are  as  follows: 

(1)  finalizing  requirements  (ongoing  definition  of  requirements  and  decision 

making  causing  delays  to  tasks  awaiting  technical  direction);  and, 

(2)  poor  requirements  definition  (incorrect  or  incomplete  definition  of 

requirements,  interfeces,  designs,  and  tasks). 

Figure  4-17  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  “technical  definition”  category. 
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FIGURE  4-17:  “Technical  Definition”  Subcategories 


“Technical  Problems”  Category.  This  category  includes  reasons  related  to 
difficulties  encountered  in  the  design  and  development  of  components  and  systems. 
Specific  subcategories  of  reasons  for  schedule  problems  within  the  “technical  problems” 
category  are  as  follows: 

(1)  analysis  problems  (difficulties  and  delays  conducting  analyses  or  studies); 

(2)  coordination  and  integration  between  system  and  related  systems; 

(3)  design  problems  (such  as  unresolved  design  issues,  design  errors,  and  design 

activities  being  more  challenging  than  expected); 

(4)  development  problems  (problems  with  acquiring  or  creating,  and  integrating 

both  hardware  and  software  into  the  system);  and, 

(5)  task  growth  (additional  effort  expended  due  to  unexpected  problems  or  the 

need  for  required  improvements). 
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Figure  4-18  presents  the  relative  contribution  of  reasons  (in  terms  of  number 
observed)  within  the  above  subcategories  to  the  'technical  problems”  category. 


FIGURE  4-18:  “Technical  Problems”  Subcategories 


A  Final  Word  on  Categorization  of  Reasons.  The  above  categories  are  useful 
for  understanding  and  making  comparisons  among  the  major  classes  of  reasons  for 
schedule  problems  on  system  development  efforts.  In  order  to  provide  further  detail,  all 
549  observed  reasons  for  schedule  problems  have  been  listed  in  the  appendix,  where  they 
are  grouped  in  categories  and  subcategories  identical  to  those  described  above.  By 
identifying  these  categories,  subcategories,  and  individual  reasons,  this  research  satisfies 
its  objective  of  identifying  the  actual  reasons  for  schedule  problems  across  large  Air  Force 
system  development  efforts,  describing  those  reasons  at  a  level  of  detail  (in  this  case,  three 
levels  of  detail)  that  will  allow  the  development  of  appropriate  corrective  actions. 

In  meeting  this  objective,  it  is  important  to  note  that  there  is  no  single  “correct” 
manner  in  which  to  classify  reasons  into  categories  and  subcategories.  Certainly,  other 
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schemes  for  defining  categories  and  assigning  reasons  to  those  categories  may  be  equally 
valid.  What  the  above  portion  of  the  analysis  provides  is  merely  the  data  presented  in  a 
reasonable,  consistent  framework  that  allows  the  above  research  objective  to  be  satisfied. 
Comparison  (by  category)  of  Reasons  for  Schedule  Problems 

The  second  objective  of  this  research  is  to  quantify  the  importance  of  each 
category  of  reasons,  in  terms  of  frequency  and  severity,  in  order  to  determine  the 
categories  of  reasons  most  and  least  deserving  of  management  attention.  To  this  end,  the 
following  subsections  quantify  and  compare  the  significance  of  the  reasons  for  schedule 
problems  in  terms  of  frequency  of  occurrence,  total  schedule  variance  (in  dollars  and  in 
work  days),  and  average  schedule  variance  (in  dollars  and  in  work  days). 

Frequency  of  Reasons  by  Category.  Figure  4-19  presents  the  total 
number  of  observed  reasons  for  schedule  problems  that  fall  within  each  category.  Based 
on  this  data,  it  appears  that  on  system  development  efforts,  certain  types  of  reasons  are 
likely  to  occur  far  more  often  than  others.  Although  frequency  is  only  one  of  several 
measures  of  reason  significance,  the  below  figure  provides  important  information  for 
determining  the  types  of  reasons  for  which  corrective  action  would  provide  the  greatest 
benefit. 
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FIGURE  4-19:  Frequency  of  Reasons  by  Category 


Total  Schedule  Variance  (in  dollars)  by  Category.  Figure  4-20  presents, 
for  each  category,  the  sum  of  the  negative  schedule  variances  associated  with  the  reasons 
assigned  to  that  category.  This  provides  a  measure  of  the  total  impact  of  the  observed 
reasons  on  the  development  efforts.  Because  the  below  schedule  variances  are  measured 
in  dollars,  they  tend  to  bias  the  results  towards  reasons  that  occur  on  larger  efforts,  which 
tend  to  experience  larger  dollar  schedule  variances.  For  a  more  complete  discussion  of 
this  topic,  please  refer  to  Chapter  3. 
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FIGURE  4-20:  Total  Schedule  Variance  (in  dollars)  by  Category 
Total  Schedule  Variance  (in  work  days)  by  Category.  Figure  4-21  presents, 
for  each  category,  the  sum  of  the  negative  schedule  variances  (in  work  days)  associated 
with  the  reasons  assigned  to  that  category.  Chapter  3  discusses  the  method  used  to 
convert  schedule  variances  from  dollar  amounts  to  fractions  of  a  month.  Within  each 
category,  then,  each  reason’s  month  fraction  was  converted  to  work  days  (assuming  an 
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average  of  22  work  days  per  month),  then  summed  to  provide  the  totals  in  the  below 
figure. 


FIGURE  4-21:  Total  Schedule  Variance  (in  work  days)  by  Category 


Like  total  schedule  variance  (in  dollars),  total  schedule  variance  (in  work  days)  provides  a 
measure  of  the  total  impact  of  the  observed  reasons  on  the  development  efforts.  As 
discussed  in  Chapter  3,  however,  schedule  variance  (in  work  days)  represents  a  month  of 
delay  on  small  efforts  the  same  as  it  represents  a  month  of  delay  on  large  efforts.  This 
tends  to  bias  the  results  towards  reasons  found  on  smaller  efforts,  which  would  tend  to  get 
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less  attention  under  normal  circumstances.  Thus,  both  types  of  schedule  variances  (in 
dollars  and  in  work  days)  provide  information  useful  in  quantifying  the  severity  of 
problems  associated  with  various  reasons. 

Average  Schedule  Variance  (in  dollars)  by  Category.  Figures  4-20  and  4-21 
quantify  the  severity  of  schedule  problems  associated  with  categories  of  reasons  for  those 
problems.  When  using  total  schedule  variances,  however,  there  is  an  underlying 
assumption  that  the  frequency  distribution  of  reasons  that  occurred  in  the  past  will  not 
change  in  the  future.  In  the  event  that  the  frequency  distribution  of  reasons  is  expected  to 
change,  the  measures  of  problem  severity  should  not  be  based  on  the  observed  (past) 
frequency  distribution.  Average  schedule  variance  per  category  provides  the  required 
separation  from  the  observed  frequency  distribution. 

Figure  4-22  presents  the  average  negative  schedule  variance  (in  dollars)  for 
reasons  within  each  category.  In  interpreting  this  figure,  it  should  be  noted  that  within 
each  category  there  is  a  wide  statistical  variance  among  the  observed  schedule  variances. 
In  fact,  across  categories,  the  standard  deviation  of  schedule  variances  observed  within  a 
category  exceeded  the  mean  value  of  the  category’s  schedule  variances  by  an  average  of 
59  percent. 
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among  the  observed  schedule  variances  (in  work  days)  within  each  category.  In  fact, 
across  categories,  the  standard  deviation  of  schedule  variances  observed  within  a  category 
exceeded  the  mean  value  of  the  category’s  schedule  variances  by  an  average  of  2  percent. 
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FIGURE  4-23:  Average  Schedule  Variance  (in  work  days)  by  Category 


Summary  of  Category  Rankings  by  Comparative  Measures.  Although  the 
above  subsections  provide  information  about  how  each  category  of  reasons  for  schedule 
problems  compares  to  other  categories  for  each  of  the  five  comparative  measures 
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(frequency,  total  schedule  variance  in  dollars,  total  schedule  variance  in  work  days, 
average  schedule  variance  in  dollars,  and  average  schedule  variance  in  work  days) 
individually,  Table  4-2  presents  the  ranking  of  each  category  of  reasons  in  terms  of  all  five 
comparative  measures  simultaneously. 


TABLE  4-2:  Summary  Ranking  of  Categories 


Category 

Frequency 

Total 

(S) 

Variance 

(Days) 

IB 

Technical  Problems 

i 

1 

3 

5 

15 

Subcontractor  Late 

2 

2 

1 

4 

6 

Manufacturing  Probs 

4 

4 

2 

7 

4 

Design  Changes 

6 

3 

5 

6 

13 

Late  Data 

3 

6 

6 

11 

16 

Contracting 

9 

. 5 

4 

1 

3 

Staffing 

4 

8 

9 

14 

19 

Gov’t  Not  Supportive 

$ 

mm 

7 

10 

Quality 

ii 

BE 

8 

2 

2 

Technical  Definition 

7 

10 

11 

13 

20 

Estimating 

10 

S 

12 

Test  Problems 

12 

13 

12 

9 

*5 

Late  Reviews 

14 

16 

i5 

16 

mm 

Changed  Plans 

g  m 

18 

15 

■Bi 

Inventory  Mgt 

R 

17 

3 

9 

Gov’t  Added  Work 

15 

17 

14 

18 

11 

Miscellaneous  Delays 

16 

15 

16 

12 

7 

Gov’t  Stopped  Work 

19 

19 

15 

17 

1 

Req'ments  Changes 

18 

18 

19 

19 

14 

Facility  Problems 

20 

17 

In  Table  4-2,  categories  have  been  ranked  from  one  to  twenty  for  each  measure  of 
schedule  problem  significance,  with  lower  numbers  indicating  the  more  significant 
categories  (categories  having  the  same  number  within  a  column  are  “tied”  for  a  given 
rank).  The  categories  have  been  presented  in  order  of  significance,  based  on  a  simple 
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aggregate  ranking  scheme,  where  the  three  rankings  to  the  left  of  the  double  line  have 
been  summed.  Again,  the  lower  the  sum  of  the  category’s  ranks,  the  more  significant  the 
category.  Note  that  this  ordering  scheme  is  merely  a  useful  tool  for  presenting  the  data, 
and  does  not  imply  that  this  scheme  is  the  “correct”  way  to  interpret  category  significance 
in  all  cases.  Rankings  to  the  left  of  the  double  line  are  those  that  should  be  considered  if 
the  frequency  distribution  of  reasons  is  not  expected  to  change  from  that  observed  in  the 
data.  If  another  frequency  distribution  of  reasons  is  expected  to  occur  m  the  future, 
rankings  to  the  right  of  the  double  line  should  be  considered,  along  with  the  expected 
future  distribution.  In  any  case,  the  rankings  in  Table  4-2  enable  informed  decisions  to  be 
made  regarding  the  categories  of  reasons  for  which  corrective  action  would  provide  the 

greatest  benefit. 

Commonality  of  Reasons  Across  System  Development  Efforts 

The  third  and  final  objective  of  this  research  was  to  demonstrate  that  the  reasons 
for  schedule  problems  are  not  program  unique,  but  are  common  across  system 
development  efforts.  The  achievement  of  this  objective  enables  schedule-related  lessons 
learned  from  past  and  present  efforts  to  influence  schedule  performance  m  future  efforts. 
Figure  4-24  demonstrates  this  commonality  of  reasons  by  presenting  the  number  of  efforts 
on  which  reasons  within  each  given  category  occurred. 
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FIGURE  4-24:  Commonality  of  Reasons  Across  Development  Efforts 


From  Figure  4-24,  it  is  clear  that  in  general,  reasons  for  schedule  problems  on 
system  development  efforts  are  not  unique  to  a  given  effort.  While  all  22  development 
efforts  did  not  experience  all  20  categories  of  reasons  for  schedule  problems,  no  category 
appeared  only  on  one  effort,  and  on  average,  categories  appeared  on  9. 1  efforts. 

Certainly,  this  is  compelling  evidence  supporting  the  further  study  of  lessons  learned  from 
past  schedule  problems  so  that  these  problems  (which  will  recur  without  intervention)  can 
be  avoided  in  the  future. 
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The  evidence  for  commonality  of  reasons  for  schedule  problems  across  system 
development  efforts  would  most  likely  be  even  stronger  if  reasons  were  classified  under 
fewer  categories.  This  approach,  however,  was  not  taken  because  in  order  to  satisfy  the 
first  two  objectives  of  this  research,  reasons  needed  to  be  explained  in  sufficient  detail 
(hence  the  need  for  a  greater,  not  a  fewer  number  of  categories)  to  allow  the  development 
of  appropriate  corrective  actions. 
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V:  Findings  and  Conclusions 


Introduction 

This  research  had  three  main  objectives.  The  first  was  to  identify  the  actual 
reasons  for  schedule  problems  across  large  Air  Force  system  development  efforts, 
describing  those  reasons  at  a  level  of  detail  that  will  allow  the  development  of  appropriate 
corrective  actions.  The  second  was  to  quantify  the  importance  of  each  category  of 
reasons,  in  terms  of  frequency  and  severity,  in  order  to  determine  the  categories  of  reasons 
most  and  least  deserving  of  management  attention.  The  third  was  to  demonstrate  that  the 
reasons  for  schedule  problems  are  not  program  unique,  but  are  common  across  system 
development  efforts,  therefore  schedule-related  lessons  learned  from  past  and  present 
efforts  are  likely  to  be  relevant  to  future  efforts.  The  following  chapter  summarizes  the 
findings  of  this  research  related  to  these  three  objectives,  comments  on  the  differences 
between  the  results  of  this  research  and  results  found  in  the  literature,  provides 
observations  regarding  the  collection  of  meaningful  schedule-related  data  from  system 
development  efforts,  and  suggests  areas  in  which  future  related  research  would  be 
appropriate. 

Findings  Related  to  the  Research  Objectives 

Identified  Reasons  for  Schedule  Problems.  In  order  to  satisfy  the  first  objective 
of  this  research,  the  reasons  for  schedule  problems  on  large  Air  Force  system  development 
efforts  were  identified.  All  549  observed  reasons  (see  Appendix)  were  categorized  (see 
Table  4-1)  and  subcategorized  (see  pages  43-61)  in  order  to  facilitate  understanding  and 
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comparing  those  reasons  at  a  level  of  detail  that  allows  the  development  of  appropriate 
corrective  actions. 

Although  the  categories  and  subcategories  are  useful  in  understanding  classes  of 
reasons  and  for  making  comparisons  among  reason  classes,  the  549  unsummarized  reasons 
contained  in  the  appendix  may  be  of  equal  or  greater  value  to  managers.  These  reasons, 
grouped  by  category  and  subcategory,  are  essentially  a  database  of  lessons  learned  from 
22  past  and  present  system  development  efforts.  The  mere  act  of  skimming  through  these 
actual  reasons  for  schedule  problems  may  provide  managers  with  the  “heads-up”  reminder 
of  things  that  have  gone  wrong  before  that  will  allow  them  to  avoid  these  problems  in  the 
future. 

Relative  Importance  of  Categories  of  Reasons.  In  order  to  satisfy  the  second 
objective  of  this  research,  the  identified  reasons  for  schedule  problems  were  compared  in 
terms  of  frequency  of  occurrence  and  schedule  variance  (in  dollars  and  in  work  days). 

The  complete  set  of  comparisons  may  be  found  in  Figures  4-19  to  4-23,  and  in  Table  4-2. 
Table  5- 1  presents  the  top  five  and  bottom  five  categories  of  reasons  in  terms  of 
frequency,  and  total  schedule  variance  (both  in  dollars  and  in  work  days).  Although 
Chapter  4  also  presented  average  schedule  variance,  in  addition  to  the  measures  presented 
in  Table  5-1,  total  schedule  variance  better  describes  the  significance  of  the  reasons  in 
terms  of  the  impact  they  actually  had  across  the  22  sampled  development  efforts. 
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TABLE  5-1 :  Most  and  Least  Significant 
Categories  of  Reasons  for  Schedule  Problems 


Top  Five  Categories 
(Rank  Ordered) 

Frequency 

Total  Schedule  Variance 
(S) 

Total  Schedule  Variance 
(Work  Days) 

Technical  Problems 
Subcontractor  Late 

Late  Data 

Manufacturing  Probs 
Staffing 

Technical  Problems 
Subcontractor  Late 

Design  Changes 
Manufacturing  Probs 
Contracting 

Subcontractor  Late 
Manufacturing  Probs 
Technical  Problems 
Contracting 

Design  Changes 

Bottom  Five  Categories 
(Rank  Ordered) 

Frequency 

Total  Schedule  Variance 

($) 

Total  Schedule  Variance 
(Work  Days) 

Miscellaneous  Delays 
Inventory  Mgt 
Req'ments  Changes 
Gov't  Stopped  Work 
Facility  Problems 

Late  Reviews 

Gov’t  Added  Work 
Req'ments  Changes 

Gov't  Stopped  Work 
Facility  Problems 

Miscellaneous  Delays 
Inventory  Mgt 

Changed  Plans 

Req'ments  Changes 
Facility  Problems 

The  seven  categories  listed  in  Table  5-1  comprising  the  “top  five,”  or  most 
significant,  categories  of  reasons  for  schedule  problems  account  for  49  percent  of  the 
observed  reasons  (frequency),  57  percent  of  the  schedule  variance  (in  dollars),  and  49 
percent  of  the  schedule  variance  (in  work  days).  Clearly,  these  categories  represent 
reasons  more  deserving  of  management  attention  than  the  eight  categories  listed  in 
Table  5-1  comprising  the  “bottom  five,”  or  least  significant,  categories  of  reasons  for 
schedule  problems,  which  account  for  only  7  percent  of  the  observed  reasons  (frequency). 
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2  percent  of  the  schedule  variance  (in  dollars),  and  8  percent  of  the  schedule  variance  (in 
days). 

Where  Management  Attention  Should  Focus.  Based  on  the  results  of  this 
research,  management  should  focus  its  attention  on  the  seven  categories  of  reasons  for 
schedule  problems  comprising  the  “top  five”  portion  of  Table  5-1.  To  this  end,  this 
subsection  presents  additional  detail  on  the  reasons  in  these  categories  and  provides 
suggestions  for  improvement. 

First,  the  “technical  problems”  category  includes  reasons  related  to  difficulties 
encountered  in  the  design  and  development  of  components  and  systems.  These  difficulties 
include:  difficulties  and  delays  conducting  analyses  or  studies;  coordination  and  integration 
between  the  system  and  related  systems;  unresolved  design  issues;  design  errors;  design 
activities  being  more  challenging  than  expected;  problems  with  acquiring  or  creating,  and 
integrating  both  hardware  and  software  into  the  system;  and,  additional  effort  expended 
due  to  unexpected  problems  or  the  need  for  required  improvements.  In  order  to  reduce 
the  number  of  “technical  problems”  encountered,  managers  should  ensure  that  adequate 
technical  planning  is  conducted  prior  to  the  start  of  a  project.  In  this  planning,  the 
integration  of  components  and  systems  should  be  given  high  priority,  because  this  aspect 
of  planning  is  often  neglected.  By  taking  a  methodical,  systems  engineering  approach  to 
project  planning,  many  (though  not  all)  “technical  problems”  can  be  eliminated. 

Second,  the  “subcontractor  late”  category  includes  reasons  related  to  late 
deliveries  or  slow  progress  by  subcontractors  or  vendors.  These  difficulties  include:  late 
deliveries  of  software,  hardware,  and  other  miscellaneous  products;  late  deliveries  by  a 
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subcontractor’s  subcontractor;  and,  slow  progress  of  a  subcontractor  toward  meeting  its 
planned  schedule.  By  being  one  of  the  top  two  categories  of  reasons  for  schedule 
problems  in  all  three  of  the  Table  5-1  measures  of  problem  significance,  subcontracting 
has  been  shown  to  be  extremely  important  to  the  schedule  success  of  defense  system 
development  efforts.  In  order  to  improve  schedule  performance  in  this  area,  managers 
need  to  increase  visibility  into  subcontractor  plans  and  procedures.  In  addition,  better 
“early  warning  systems”  are  needed  to  allow  managers  to  predict  subcontractor  delays  in 
time  to  implement  alternate  work  plans. 

Third,  the  “manufacturing  problems”  category  includes  reasons  related  to  problems 
building  hardware,  both  in  the  development  and  preparation  for  production  of  a  system 
These  problems,  encountered  in  translating  an  engineering  design  into  developmental  and 
production  hardware,  include:  late  requisitioning  of  inventory;  slowdowns  due  to  new 
computer  systems;  design  problems  discovered  during  fabrication;  fabrication  problems  on 
breadboards,  system  components,  tooling,  and  test  equipment;  late  receipt  of  material 
impacting  fabrication  and  assembly;  late  start  of  tooling;  manufacturing  design  problems  in 
developing  the  bill  of  materials,  releasing  tooling,  and  providing  articles  required  for 
development;  machine  proofing  problems  and  delays  impacting  fabrication  and  assembly; 
and,  manufacturing  process  problems,  such  as  fabrication  processes  that  produce  system 
components  that  do  not  meet  the  specification.  Just  as  in  the  “technical  problems” 
category,  much  improvement  could  occur  in  the  “manufacturing  problems”  area  if 
management  required  a  methodical,  integrated,  systems  engineering  approach  to  planning 
these  activities. 
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Fourth,  the  “design  changes”  category  includes  reasons  related  to  changes  in 
system  or  component  designs,  typically  undertaken  to  fix  problems  or  to  improve 
performance.  Quite  often,  in  addition  to  requiring  time  to  implement,  these  changes  also 
impact  other,  related  tasks  that  depend  on  stable,  defined  designs  for  their  continued 
progress.  The  key  to  mitigating  problems  in  this  area  is  for  management  to  be  aware  that 
design  changes  often  have  a  “ripple  effect”  throughout  the  system  Before  permitting  a 
design  change,  managers  should  insist  on  a  thorough  analysis  of  the  potential  impacts  on 
the  entire  system 

Fifth,  the  ‘late  data”  category  includes  reasons  related  to  the  late  receipt  or 
generation  of  required  information  either  within  the  contractor’s  organization,  or  between 
contractor  and  subcontractor.  These  difficulties  include:  late  completion  of  specifications 
and  other  documents  required  for  delivery  to  the  government;  late  engineering  release  of 
drawings  and  other  design  data  required  for  manufacturing  and  other  activities;  and,  late 
or  incomplete  information  (such  as  specifications,  reports,  and  data)  impacting  areas  such 
as  design,  manufacturing,  training,  provisioning,  technical  publications,  test,  facilities,  and 
material  orders.  The  lesson  of  this  category  is  that  system  development  is  a  data  intensive 
activity,  and  that  efforts  to  speed  the  generation  and  flow  of  data  will  speed  the  overall 
development  effort.  Plans  that  incorporate  the  need  for  data  generation  and  sharing, 
procedures  that  facilitate  these  actions,  and  computer  based  tools  that  enable  these  actions 
can  all  have  a  significant  positive  impact  on  development  effort  schedule  performance. 

Sixth,  the  “contracting”  category  includes  reasons  related  to  contractual  actions 
and  the  process  of  awarding  subcontracts.  These  difficulties  include:  source  selection  of 


76 


subcontractors  and  vendors  (including  request  for  proposal  (RFP)  preparation  and 
proposal  receipt);  placing  subcontractors  and  vendors  on  contract;  processing  purchase 
orders;  and,  terminating  subcontractors.  Improvements  in  this  area  could  be  realized  by 
thoroughly  planning  subcontracting  activities  at  the  beginning  of  the  development  effort, 
and  by  further  streamlining  source  selection  procedures. 

Seventh,  the  “staffing”  category  includes  reasons  related  to  having  insufficient 
personnel  assigned  to  tasks.  These  difficulties  include:  hiring  delays,  either  during  the 
initial  ramp-up  of  personnel  to  conduct  the  effort,  or  in  replacing  personnel  later  in  the 
effort;  inadequate  staffing  for  the  timely  completion  of  tasks  either  at  contractor  or 
subcontractor  facilities;  receipt  of  security  clearances  taking  longer  than  expected,  causing 
delays  until  sufficient  numbers  of  cleared  personnel  are  available  to  work  on  the  effort; 
reassignment  of  personnel  to  higher  priority  or  nearer  term  tasks,  resulting  in  delays  on  the 
original  tasks;  and,  having  the  wrong  people  assigned  to  a  task,  such  as  when 
subcontractors  with  either  a  lack  of  required  expertise  or  with  an  inappropriate  mix  of 
personnel  are  working  on  a  task.  Improvements  in  this  area  could  be  realized  by  locating 
appropriate  personnel  prior  to  contract  award,  planning  for  an  adequate  number  of 
personnel  to  meet  the  schedule  demands  of  the  effort,  ensuring  that  the  qualifications  of 
personnel  are  matched  to  the  tasks  they  are  to  perform,  and  allowing  adequate  time  to 
both  locate  replacement  personnel  and  to  process  security  clearances. 

The  preceding  suggestions  for  improvement  only  “scratch  the  surface”  of  potential 
corrective  actions  that  could  be  applied  to  improve  schedule  performance  on  defense 
system  development  efforts.  Indeed,  a  thorough  investigation  to  identify  corrective 
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actions  based  on  the  reasons  for  schedule  problems  identified  in  this  thesis  would  be  a 
significant  research  effort  in  and  of  itself.  Hopefully,  this  research,  by  providing  a  starting 
point  in  terms  of  identifying  the  reasons  for  schedule  problems  and  suggesting  which 
categories  of  reasons  most  require  management  attention,  will  enable  both  researchers  and 
practitioners  to  develop  appropriate  corrective  actions  to  mitigate  schedule  problems  in 
the  future. 

Commonality  of  Reasons  Across  Development  Efforts.  In  order  to  satisfy  the 
third  and  final  research  objective,  the  number  of  development  efforts  on  which  each 
category  of  reasons  for  schedule  problems  had  been  observed  was  analyzed  to 
demonstrate  that  reasons  were  not  unique  to  a  given  effort.  According  to  this  analysis,  as 
presented  in  Figure  4-24,  although  all  22  development  efforts  did  not  experience  all  20 
categories  of  reasons  for  schedule  problems,  no  category  appeared  only  on  one  effort,  and 
on  average,  categories  appeared  on  9. 1  efforts.  Certainly,  this  is  compelling  evidence 
supporting  the  further  study  of  lessons  learned  from  past  schedule  problems  so  that  these 
problems  (which  will  recur  without  intervention)  can  be  mitigated  in  the  future. 
Differences  Between  the  Results  of  this  Research  and  Results  in  the  Literature 

Overall,  there  is  a  remarkable  similarity  between  the  reasons  for  schedule  problems 
found  in  the  literature  (see  Tables  2-1,  2-2,  and  2-3),  and  those  observed  in  the  course  of 
this  research  (see  Table  4-1,  pages  43-61,  and  Appendix).  The  few  exceptions  are  as 
follows. 

First,  several  reasons  for  schedule  problems  mentioned  in  the  literature  were  not 
observed  in  the  data.  One  explanation  for  these  discrepancies  is  that  the  data  source  (Cost 
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Performance  Reports,  or  CPRs)  is  written  from  a  contractor  perspective,  whereas  some  of 
the  reasons  found  in  the  literature  are  stated  from  a  government  perspective.  For  example, 
poor  management  practices,  a  lack  of ‘follow-up,”  inadequate  supervision,  and  a  lack  of 
motivation  are  all  things  that  a  contractor  is  unlikely  to  admit  on  a  report  being  sent  to  the 
government  (CPR).  Similarly,  there  would  be  a  tendency  to  downplay  the  effects  of 
program  manager  turnover,  and  not  to  address  political  influences  or  economic  factors  on 
a  CPR  In  addition,  there  would  be  a  tendency  to  not  be  too  critical  of  the  government, 
avoiding  references  to  micromanagement,  constructive  changes,  and  problems  with  joint 
service  project  management.  Finally,  Engineering  Change  Proposals  (ECPs)  would 
probably  not  be  listed  as  a  reason  for  schedule  problems  on  a  contractor  report  because 
when  ECPs  modify  the  work  to  be  accomplished,  they  also  modify  the  baseline  schedule 
against  which  progress  on  the  development  effort  is  measured.  Thus,  although  the  ECP 
may  lengthen  the  development  effort,  it  doesn’t  typically  cause  a  schedule  variance  (and 
thus  is  not  reported)  unless  delays  occur  in  conducting  the  work  associated  with  the  ECP. 

Another  explanation  for  reasons  not  appearing  in  the  data  is  that  some  reasons  in 
the  literature  may  be  reasonably  rare,  and  therefore  were  not  observed  in  the  limited 
sample  of  development  effort  reports  used  in  this  research.  Reasons  such  as  labor 
problems,  concept  stability,  and  external  events  may  fall  into  this  category. 

Second,  several  reasons  for  schedule  problems  observed  in  the  data  were  not 
mentioned  in  the  literature.  These  reasons,  such  as  contractual  actions,  changed  plans, 
facilities  problems,  late  direction,  technical  definition,  manufacturing  problems,  and  test 
problems,  tend  to  be  more  specific  than  many  of  the  reasons  in  the  literature.  This  is  to  be 
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expected  in  an  initial  descriptive  study  such  as  this  one,  where  the  desired  outcome  is  to 
identify  reasons  not  already  expressed  and  to  provide  added  detail  such  that  the  identified 
reasons  are  useful  to  managers  for  developing  appropriate  corrective  actions. 

Third  and  finally,  unlike  the  reasons  provided  in  the  literature,  the  reasons  for 
schedule  problems  identified  in  this  research  have  been  quantified  in  terms  of  the 
significance  of  their  associated  schedule  problems  so  that  managers  can  determine  the 
categories  of  reasons  most  and  least  deserving  of  their  attention. 

Obtaining  Meaningful  Schedule-Related  Data  from  System  Development  Efforts 

Three  observations  regarding  obtaining  meaningful  schedule-related  data  from 
large  Air  Force  system  development  efforts  are  worthy  of  mention.  First,  it  is  difficult  to 
find  a  usable  source  of  this  type  of  data.  In  the  Air  Force,  and  in  the  DoD  in  general, 
schedule  has  been  of  significantly  less  interest  than  cost.  This  is  reflected  in  the  fact  that 
although  there  are  numerous  functional  organizations  throughout  the  DoD  that  focus  on 
cost,  there  are  few  if  any  that  focus  primarily  on  schedule.  The  result  is  that  unlike  cost 
information  schedule  information  is  rarely  archived  and  seldom  studied.  Fortunately,  due 
to  the  interest  of  the  cost  community,  the  Cost  Performance  Reports  (which  also  happen 
to  include  schedule  information)  were  a  readily  available  source  of  data  for  this  research. 

Tn  order  to  properly  preserve  the  schedule-related  lessons  of  the  past  so  that  mistakes  are 
not  continually  repeated,  more  schedule-related  information  needs  to  be  archived  and 
studied  in  the  future. 

Second,  in  general,  the  format  of  Cost  Performance  Report  (CPR)  has  not  been 
optimized  for  identifying  and  quantifying  the  reasons  for  schedule  problems.  This  is  partly 
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because  the  format  of  the  CPR  varies  from  contract  to  contract.  Of  the  22  development 
efforts  examined,  many  had  CPRs  formatted  such  that  reasons  were  easy  to  identify  and 
quantify,  however  others  made  identification  and  quantification  more  difficult  by 
presenting  poorly  organized,  rambling  narratives,  and  by  grouping  many  reasons  in  a 
narrative  without  identifying  the  schedule  variance  associated  with  each  reason.  To 
remedy  this  problem,  guidelines  could  be  modified  to  call  for  contractors  to  explain 
variances  in  a  way  such  that  the  root  reasons  for  schedule  problems  are  clearly  and 
succinctly  identified,  followed  by  whatever  narrative  is  necessary  to  understand  the  nature 
of  the  problems  and  associated  consequences.  Also,  if  a  reportable  schedule  variance  has 
several  reasons,  contractors  should  identify  the  amount  of  variance  associated  with  each 
reason  (as  some  contractors  already  do). 

Third  and  finally,  there  needs  to  be  a  better  measure  than  schedule  variance  for 
quantifying  the  severity  of  schedule  problems.  As  explained  in  Chapter  3,  negative 
schedule  variance  is  based  on  a  deviation  from  a  time-phased  budget,  and  while  it  is  a 
reasonable  indicator  of  schedule  inefficiency,  it  does  not  usually  equate  with  delay  to  the 
development  effort.  Any  measure  that  more  directly  describes  the  delay  to  the  overall 
effort,  or  at  least  the  loss  of  “slack  time”  on  tasks  would  be  an  improvement  over  the 
current  schedule  variance  measure  because  it  would  better  describe  the  real  world 
management  concern  of  project  delay. 

Suggested  Areas  for  Future  Research 

Based  on  the  experience  gained  and  the  results  provided  in  this  research,  the 
following  three  areas  are  suggested  for  fixture  exploration.  First,  as  was  stated  earlier  in 
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this  chapter,  the  data  source  used  in  this  research  (the  Cost  Performance  Report)  is  written 
from  a  contractor  perspective.  In  order  to  gain  additional  insight  into  the  reasons  for 
schedule  problems  on  system  development  efforts,  it  is  highly  desirable  to  also  use  a  data 
source  written  from  a  government  perspective.  The  Defense  Acquisition  Executive 
Summary  (DAES)  report  is  such  a  data  source,  which  is  similar  to  the  CPR,  yet  is  written 
by  the  government  manager  of  the  system  development  effort.  The  DAES  reports  are 
centrally  archived  by  the  Performance  Management  Branch  of  the  Office  of  the 
Undersecretary  of  Defense  for  Acquisition  and  Technology  in  the  Pentagon.  The  only 
complication  with  using  the  DAES  as  a  data  source  is  that  portions  of  the  DAES  are 
classified.  Although  the  data  required  to  support  further  research  into  the  reasons  for 
schedule  problems  on  system  development  efforts  are  generally  not  classified,  the  data 
would  still  have  to  be  extracted  from  the  classified  reports.  Once  extracted,  however,  the 
data  would  allow  a  future  effort  to  balance  the  contractor  perspective  contained  in  this 
research  with  a  much  needed  government  perspective. 

Second,  this  research  has  provided  a  great  deal  of  information  about  the  reasons 
for  schedule  problems  on  past  and  present  large  Air  Force  system  development  efforts  that 
can  be  used  to  help  avoid  these  types  of  problems  in  the  future.  This  information 
represents  a  host  of  lessons  learned  from  previous  efforts  that  provide  managers  with 
valuable  insight  into  the  types  of  problems  that  actually  occur  across  development  efforts. 
Given  the  value  of  these  insights,  it  is  important  that  this  research  approach  not  be  limited 
to  schedule-related  problems.  In  fact,  a  very  valuable  contribution  would  be  to  apply  the 
general  approach  of  this  research  to  the  cost  data  contained  in  Cost  Performance  Reports. 
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This  would  allow  managers  a  better  understanding  of  the  reasons  for  cost  problems  on 
system  development  efforts,  which  in  general  tend  to  be  even  more  significant  than 
schedule  problems. 

Third  and  finally,  because  of  the  lack  of  existing  information  regarding  the  reasons 
for  schedule  problems  on  system  development  efforts,  this  research  necessarily  focused  on 
describing  the  reasons,  rather  than  on  determining  causal  relationships  for  explaining  why 
reasons  occurred  as  they  did.  Now  that  this  research  provides  some  insight  into  the 
reasons  for  schedule  problems,  future  research  can  focus  on  specific  management 
questions.  For  example,  it  may  be  interesting  to  investigate  how  reasons  vary  from  one 
development  effort  type  (such  as  fighter  aircraft,  or  simulators)  to  another.  Perhaps 
certain  types  of  reasons  are  more  likely  to  appear  on  certain  types  of  development  efforts. 
Also,  it  may  be  useful  to  investigate  how  reasons  vary  from  one  timeframe  to  another. 
Perhaps  changed  management  practices,  such  as  total  quality  management  (TQM)  or 
acquisition  reform,  impact  the  types  and  quantities  of  reasons  for  schedule  problems  that 
are  experienced  from  one  year  to  the  next.  In  such  future  research  efforts,  great  care 
would  have  to  be  taken  to  control  for  differences  in  potential  moderating  variables  such  as 
development  effort  size,  type  and  level  of  completion  (for  example,  it  is  important  to  avoid 
comparing  efforts  early  in  development  with  those  late  in  development).  Also,  if  reasons 
are  being  compared  among  timeframes,  it  is  important  to  ensure  that  data  is  sampled 
equivalently  among  those  timeframes.  Given  the  need  for  careful,  tailored  data  collection 
on  these  types  of  future  research  efforts,  the  more  general  research  contained  in  this  thesis 
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will  most  likely  not  directly  apply,  although  it  will  certainly  provide  a  source  of  future 
research  questions  and  a  context  within  which  to  assess  future  research  results. 

Conclusion 

This  research  has  filled  a  gap  in  the  existing  management  body  of  knowledge 
regarding  the  reasons  for  schedule  problems  on  system  development  efforts.  By  capturing 
the  schedule-related  lessons  of  past  and  present  development  efforts,  and  presenting  them 
in  a  coherent  framework,  these  lessons  of  yesterday  can  be  used  by  the  managers  of  today 
to  solve  the  schedule  problems  of  tomorrow.  This  research  is  not  only  valuable  by  itself, 
but  provides  a  starting  point  from  which  more  specific  schedule-related  management 
questions  can  be  addressed  in  the  future. 
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Appendix 


549  Reasons  for  Schedule  Problems 
Observed  on  22  Large  Air  Force  System  Development  Efforts 
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Reason  for  Schedule  Variance _ |  ($000)  |  (Days)  Program  |Year 
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Category _ Reason  for  Schedule  Variance _  ($000)  |  (Days)  |  Program  |Year 
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Variance  Variance 

Category _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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Category _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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Variance  Variance 

Category  1 _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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Variance  Variance 

Category  _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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Category  I _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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-  Variance  Variance 

Category  _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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Category _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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Air  data  computer  sys  failure  to  pass  test  (sub)  1 285  15.45  Aircraft/Missile  1 

Mechanical  fit  problem  at  sub  discovered  after  PDR  55  0.22  Aircraft/Missile  2 

Rejected  H/W  delivery  due  to  excessive  test  time  on  components  98  22.00  Aircraft/Missile  4 

Rejection  of  subcontractor  hardware  1 78  39.96  Aircraft/Missile  4 

Subcontractor  components  required  additional  debug  &  integration _ _ 61  1.74  [Simulator  2 


Category _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 


*  *  i.  2  2 

5  fe  t  =>  => 

ro  m  o-  it  «t 

3-5  LU  «  « 

EE  O  g  g 

W  CO  <  <  < 


9  5  5  8  S 
5  d  ®  2 


T-  CO  ^  to  to 
3  3  *c  'g  0)  ©  <p 

S  5  5  S  «  B  » 

EEEEwwaoO 

aaacLir^icJ 


LU  LU  LU  HI  JO  A  23 

OOOOgggE 
<$<<<<  <W 


IS  «  8 

o  <d  o 


c  c  c  c  c 

(D  (D  0)  ©  © 

E  E  E  E  E 

a.  q.  9.  a.  q. 

=3  '3  *3  3  3 

cr  0-  cr  cr  or 

LU  LU  LU  LU  LU  I 

O  O  O  O  O  | 

<  <  <  3  <  < 


5  8  2  8  S  ! 

oi  in  cn  cn  1 


8S  §§£ 


8  8  S  ?  8  3  5 


© 

o 

.1  I 

III 

2  a  ® 
CL  -O  ^ 

8*  g  f 

CD  |  a- 
£  -6  <d 
^  <5  £ 


ill 
1 1 1 
I  €  g 

E  cl  o 
81  | 
I  s  § 

c  «  .0 

©  o  3 

>  a  co 


5  i2 

cr  c 
u>  ©  © 
«  £  E 
©  *0  © 

E  8  I 

©  o  2. 

•I  -q  sr 


©  ©  y= 

£  E  o 

©  A  CO 

2  .e  O 

3  -  » 

-0  ©  © 

c  O)  O) 

«5  c  c 
o.  cc  a 
S'  x:  .c 

LU  O  O 


cue 

«  ©  E? 
«  >,  © 
£  o> 
>  a  c 

5  3  ’o 

» 5  §- 

o  I  -E 

*  8.Z 

1  5  £ 
§  s  » 
£  §  is 

3  .9  ® 
CO  CO  Q 


5  I- 

<=  -  ® 

I  Si 

*0  ©  fc 
.1  -g  t» 

D  ®  C 


©  o  'p- 

s-  g  » 


^  O)  E 

i  t  -£ 

©  o  £ 

*  S  " 

©  w  © 
A  ©  CO 
3  S  3 

cr  o  o' 

©  CL  © 
T3  C  T3 
AAA 


8. 

at  ® 

C  © 

|  Q 

C  O) 

C 


98 


Understaffing  in  test  engineering  design  area  31  5.25  A/C  Equipment  1 

Shortage  of  manpower  in  shelter  design  6  8.25  A/C  Equipment  2 

Staffing  shortages  in  H/W  and  S/W  engineering  237  3.32  A/C  Equipment  2 

Manpower  shortages  266  4.71  A/C  Equipment  2 

Loss  of  software  engineers _  16  j  3.12  A/C  Equipment  3 


Category _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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Variance  Variance 

Category _ _ Reason  for  Schedule  Variance _ ($000)  (Days)  Program  Year 
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Category  | _ Reason  for  Schedule  Variance _  ($000)  (Days)  Program  Year 
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S/W  requirements  not  finalized  16  2.21  A/C  Equipment  3 

Subktr  delays  with  top  level  design  28  2.21  Aircraft  Upgrade  1 

Lack  of  flight  test  instrumentation  definition  13  1.75  Aircraft  Upgrade  1 

Vendor  negotiations  on  spec  control  dwg  defining  antenna  footprint _ _ 1 6  0.42  Aircraft  Upgrade  2 
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Delay  in  related  system's  spec  delayed  S/W  spec  approval  (mtej 
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Delayed  detailed  development  of  subsystem  algorithms  by  subcontractor  18  0.79  Aircraft  Upgrade  5 

Avionics  simulation  effort  suspended  due  to  mission  computer  delays  293  2.57  Aircraft/Missile  1 
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Mockup/aircraft  inconsistencies  caused  redesign  (integration  problem)  71  8.40  Aircraft/Missile  4  1989 

Weight  avoidance  efforts  delaying  drawing  release  69  1 .42  Aircraft/Missile  5  1982 

Non  completion  of  operational  flight  program  SAA/  recode  effort  94  9.53  Aircraft/Missile  6  1987 

Unanticipated  rework  of  graphics _ 68 _ 5.50  Simulator  2 _ 1991 
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